Commit Graph

28 Commits (18785bb297bf4c3db076e5ae10a9c42edebde59f)

Author SHA1 Message Date
Garrett Beaty 51d99b2f3a Fix the use of raw_io output placeholders.
In order to enforce that the test data provided for placeholders is of
the appropriate type (text vs binary), existing mismatches must be
resolved. This will prevent type mismatch issues from occurring when
switching builders to use python3.

Bug: 1273548
Change-Id: I83da25432b67853992d7771aba558fd38520a24e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3300310
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Garrett Beaty <gbeaty@google.com>
3 years ago
Gavin Mak 76b679fcf2 Add PYTHON_VERSION_COMPATIBILITY to gclient recipe module
Bug: 1227140
Change-Id: I18ad289ffac1df5482a3d6a6e4484bd69a6d6929
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3123938
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Anthony Polito <apolito@google.com>
4 years ago
Gavin Mak 56c8a27430 Add python3 support to gclient recipe module
Recipe-Nontrivial-Roll: build
Bug: 1227140
Change-Id: Ia29c154453b40015359e5ac93a533e0dd40e5c55
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3012236
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Anthony Polito <apolito@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
4 years ago
Edward Lesmes 7522924749 gclient: Set DEPOT_TOOLS_REPORT_BUILD to report bot metrics.
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: chromiumos
Change-Id: I5c7b2188a84580a0575e9e748946e66cc7140787
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2872856
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Quinten Yearsley b674f8a277 Edit depot_tools recipe_modules docstrings for consistency
Change-Id: I454e9c90d3159598ae34d97d22c96b9d20d55f80
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2521656
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
5 years ago
Greg Guterman 71fd8abb9d Add explicit "python" to script for Windows compatibility
Remove cwd augmentation for good measure. This should be done by the
caller.

Bug: 923016
Change-Id: I7f6618d7a9e56f70168fc71046412c6d462f40fb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2311320
Auto-Submit: Gregory Guterman <guterman@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Greg Guterman c3cb7eecb6 Analyze DEPS autorolls
This new logic tries to figure out which files really changed in a DEPS roll by checking out the old DEPS file and recursively running git diff on all of the repos listed in DEPS. When successful, affected_files turns from just ['DEPS'] into an actual list of affected files. When not successful, it falls back to the original logic.

When DEPS is autorolled, typically the change is just an updated git revision. We don't apply this logic to manual DEPS rolls, as they can contain changes to other variables, whose effects are not easily discernible.

MUST be applied in tandem with https://crrev.com/c/2250928

Bug: 923016
Change-Id: I9e33461ea4ae81dc056d229f8604943745e30389
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2283855
Commit-Queue: Gregory Guterman <guterman@google.com>
Reviewed-by: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Robert Iannucci 4504fd580a Remove infra_paths module.
We no longer need it to bridge path differences between buildbot and
LUCI, and it shouldn't be in use downstream now either.

R=machenbach@chromium.org, martiniss@chromium.org, tandrii@chromium.org, yiwzhang@google.com

Bug: 1015181
Change-Id: Iaba2100653ae9f06799d1d510bef993f7f710dc4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2220171
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
5 years ago
Garrett Beaty efd15ecf30 Avoid accessing the None json output on gclient sync failure.
Change-Id: Ib6bab87bf2b98b5122c56d37218c7f6db1bfdfe2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1838116
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
6 years ago
Marc-Antoine Ruel 8e57b4bc55 python3 improvements
Ran:
  vi $(git grep --name-only iteritems | grep -v third_party)
  vi $(git grep --name-only itervalues | grep -v third_party)
  vi $(git grep --name-only 'print ' | grep -v third_party)

and edited the files quickly with adhoc macros. Then ran in recipes/:
  ./recipes.py test train

There was only a small subset of files that had been updated to use
six.iteritems() and six.itervalues(). Since the dataset size that is
being used in gclient is small (pretty much always below 200 items),
it's better to just switch to .items() right away and take the temporary
performance hit, so that we don't need to come back to rewrite the code.

Recipe-Nontrivial-Roll: build
Bug: 984182
Change-Id: I5faf11486b66b0d73c9098ab0f2ce1b15a45c53e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1854900
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Marc-Antoine Ruel <maruel@chromium.org>
6 years ago
Andrii Shyshkalov 2025b98401 recipes: update to stop using legacy properties.
The only exception is 'revision' of gclient's weird RevisionResolver,
which will be tackled in https://crbug.com/949334

R=ehmaldonado

Bug: 877161
Change-Id: Ic65a2d13578802cf093f2a1cfb4002fe97509b3b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1818973
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
6 years ago
Robert Iannucci efc7e91fa7 [recipes] Manual roll.
Update "import DEPS" syntax and package_repo_resource.

recipe_engine:
  https://crrev.com/a42dce4e0b0db8bd6b7688030e723197919306db [engine] Rewrite/simplify loading infrastructure. (iannucci@chromium.org)

R=jchinlee@chromium.org, martiniss@chromium.org, tandrii@chromium.org

Recipe-Manual-Change: skia
Recipe-Manual-Change: skiabuildbot
Recipe-Manual-Change: build
Recipe-Manual-Change: build_limited_scripts_slave
Recipe-Manual-Change: infra
Recipe-Manual-Change: release_scripts
Bug: 914992
Change-Id: Ie3a3be9124e1ca4abd5e79af2f8c946b3ce906ff
Reviewed-on: https://chromium-review.googlesource.com/c/1464761
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
6 years ago
Nodir Turakulov 79c651330b [gclient] Remove is_blink_mode
Not used

R=tandrii@chromium.org

Bug: 694348
Change-Id: I3188881b1f92df1cbfdbf3aafe309abb9488d9b2
Reviewed-on: https://chromium-review.googlesource.com/1232874
Auto-Submit: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
7 years ago
Nodir Turakulov 53e28ecb9f [gclient] Remove calculate_patch_root
It is not used. get_gerrit_patch_root is used instead.

Bug: 694348
Change-Id: Icc717e38cbcaa8050c8ce8a364202b81d7f9cf2c
Reviewed-on: https://chromium-review.googlesource.com/1231282
Auto-Submit: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
7 years ago
Nodir Turakulov dd2ce986ce [gclient] Update set_patch_project_revision
Update set_patch_project_revision to use repo_path_map instead of
patch_projects.
Also rename to set_patch_repo_revision.

Bug: 694348
Change-Id: Iba31f4910f8282160000b8fcef197052597530a4
Reviewed-on: https://chromium-review.googlesource.com/1226107
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Nodir Turakulov 7b54572eb0 [gclient] Fix typo in get_gerrit_patch_root
https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1217670/17
used solution instead of solution name.

TBR=tandrii@chromium.org

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Bug: 694348
Change-Id: I829cba9223d73383791b018f9a23598443e8fa2c
Reviewed-on: https://chromium-review.googlesource.com/1231278
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
7 years ago
Nodir Turakulov 0f322bfcf5 [bot_update] Add gclient.get_gerrit_patch_root
Add a shortcut for getting local path to the patch repo.

In bot_update.api._destination_branch, start using it instead of reading
gclient config directly. This removes one more patch_project usage.

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Bug: 694348
Change-Id: If05371d0f6d11be6803ffa8392e029f5860dda25
Reviewed-on: https://chromium-review.googlesource.com/1217670
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Nodir Turakulov 4e3920bef0 [gclient] Start actively using repo_path_map
Use repo_path_map in all places where patch_projects is currently used.
Add gclient.get_repo_path function to resolve path to a repo given its URL.

infra.git requires a non-trivial roll because of incorrect expectations.

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Bug: 877161
Change-Id: Iaea0a3f14609bf0c18d4ecdf8564153b8052584d
Reviewed-on: https://chromium-review.googlesource.com/1208363
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Sergiy Byelozyorov d38a2da9f1 Use cwd from context instead of kwargs
It is not possible to use cwd argument, since none of the methods called by this
module define cwd explicitly and low-level modules like api.step do not support
it anymore, resulting in a TypeError exception thrown due to unexpected keyword
argument, e.g. see https://ci.chromium.org/swarming/task/3e74a52905a71110. Given
that any users using cwd argument would be broken anyway, it should be safe to
make this change.

Planned usage: https://crrev.com/c/1122874

R=tandrii@chromium.org, iannucci@chromium.org

Bug: 793687
Change-Id: Iadf6bf91d8d41dbc6751a848bb69d7922a206ade
Reviewed-on: https://chromium-review.googlesource.com/1122873
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
7 years ago
Andrii Shyshkalov cd8d4a659a gclient recipe: use env_suffixes insetad of custom ENV manipulation.
R=iannucci@chromium.org

Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: release_scripts
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Change-Id: Ie0b042389b7e2c505cd8dbecc1a30a37244ae238
Reviewed-on: https://chromium-review.googlesource.com/1100110
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Michael Moss 210ed0ebfe Expose new gclient.sync() arg in gclient.checkout().
R=phajdan.jr@chromium.org

Change-Id: I7313b8a26a5f2934e26491fc4b9403ba6cea0cee
Reviewed-on: https://chromium-review.googlesource.com/691017
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Commit-Queue: Michael Moss <mmoss@chromium.org>
8 years ago
Aaron Gable a16ccaf11a bot_update: use patch repo instead of project if it is mapped
This is a step away from patch_project, which was a holdover
from Rietveld. Instead of mapping "projects" (which are
poorly-defined and not guaranteed to be unique) to subpaths,
instead map repository urls (which are at least unique).

Eventually we may be able to compute this directly from DEPS
instead of hardcoding the mapping here, but at least this is
a step in the right direction.

Bug: 765633
Change-Id: Idd65984fc6edefcbedb0438d38c2338b10b7e8e5
Reviewed-on: https://chromium-review.googlesource.com/690776
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
8 years ago
Michael Moss 2da7cbd767 Add ability to pass extra flags to gclient.sync().
R=agable@chromium.org, phajdan.jr@chromium.org

Change-Id: Id883945d61aabf90e11368fbf603196e62ff0f52
Reviewed-on: https://chromium-review.googlesource.com/688675
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Commit-Queue: Michael Moss <mmoss@chromium.org>
8 years ago
Michael Moss ced744fa46 Remove unused gclient.sync() 'with_branch_heads' arg.
This arg used to be used in a 'sync' call that was removed in
crrev.com/7090f652d180eeff2663e3e3091d5db14123284c, but it was never
applied to the remaining 'sync' call. This removes the arg and leaves
"--with_branch_heads" (and adds "--with_tags") as default sync flags.

Change-Id: Ice5a592d95a0b09dafddab24769f5edd8835a1c9
Recipe-Manual-Change: build_limited_scripts_slave
Recipe-Manual-Change: build
Reviewed-on: https://chromium-review.googlesource.com/677754
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Commit-Queue: Michael Moss <mmoss@chromium.org>
8 years ago
Robert Iannucci 5702f0baef [recipes] convert all recipe_modules to use new context module.
R=recipe-roller@chromium.org

Bug:
Change-Id: I84c19965b779f1010c00cd8c857b7533eec7413f
Reviewed-on: https://chromium-review.googlesource.com/502929
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
8 years ago
Michael Achenbach c180dc4030 Remove directly accessing got_revision_mapping
Bug: 713356
Change-Id: I2c5d92152b293ca553efc875f49c28fb0fc888ed
Reviewed-on: https://chromium-review.googlesource.com/484460
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
8 years ago
Michael Achenbach 93f69da3b2 Support merging got_revision mappings during transition phase.
A few rare recipes use two or more solutions side-by-side.

This allows switching one solution in isolation to use the new mapping.

Bug: 713356
Change-Id: Id51f2f777681401843004577a4de65a6c5d1e4a6
Reviewed-on: https://chromium-review.googlesource.com/484341
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Paweł Hajdan, Jr f14709556d Move recipes to their own subdirectory
This will help remove confusion when depot_tools' recipes.py
ends up in PATH.

BUG=699120

Change-Id: Id4c21b0cc6bb022ea2c21145abe76bebb0a8d9c1
Reviewed-on: https://chromium-review.googlesource.com/458430
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
8 years ago