Commit Graph

20 Commits (main)

Author SHA1 Message Date
Alex Kravchuk 4d581d7b3f Revert "Add parse_commit_position parameter to BotUpdateApi.ensure_checkout."
This reverts commit 66b3972fc5.

Reason for revert: This won't work because chromium_checkout relies on the position being set in the output commit in order to set information for RDB. We'll have to come up with a different solution.

Original change's description:
> Add parse_commit_position parameter to BotUpdateApi.ensure_checkout.
>
> When true and got_revision_cp is present, output commit ref and position are set from got_revision_cp. Default value is true.
>
> For Chrome builds that build from tags we will set parse_commit_position to false to show tags instead of branches on Buildbucket UI, e.g. refs/tags/132.0.6824.0 instead of refs/branch-heads/6824@{#1}.
>
> Bug: 366409421
> Change-Id: I77d01615edb6b791445a06469f80c673c97ad8d6
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6000539
> Reviewed-by: Garrett Beaty <gbeaty@google.com>
> Reviewed-by: Scott Lee <ddoman@chromium.org>
> Commit-Queue: Alex Kravchuk <alexanderkr@google.com>

Bug: 366409421
Change-Id: I21a5fc6e011a64fffd1d2ef0689d5a3f3843482c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6037603
Reviewed-by: Garrett Beaty <gbeaty@google.com>
Commit-Queue: Alex Kravchuk <alexanderkr@google.com>
Reviewed-by: Scott Lee <ddoman@chromium.org>
8 months ago
Alex Kravchuk 56d4cf54ae Change BotUpdateApi.ensure_checkout to only have an option to skip commit position parsing for tags.
Rename parse_commit_position parameter to parse_commit_position_for_tags. If input commit ref is not a tag, or parse_commit_position_for_tags is True, execute the old code path. This makes the changes backward compatible with the existing code.

Bug: 366409421
Change-Id: I58405325e406a82c2e255fe5d3d4a2883c98a84a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6035030
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Alex Kravchuk <alexanderkr@google.com>
Reviewed-by: Garrett Beaty <gbeaty@google.com>
8 months ago
Alex Kravchuk 66b3972fc5 Add parse_commit_position parameter to BotUpdateApi.ensure_checkout.
When true and got_revision_cp is present, output commit ref and position are set from got_revision_cp. Default value is true.

For Chrome builds that build from tags we will set parse_commit_position to false to show tags instead of branches on Buildbucket UI, e.g. refs/tags/132.0.6824.0 instead of refs/branch-heads/6824@{#1}.

Bug: 366409421
Change-Id: I77d01615edb6b791445a06469f80c673c97ad8d6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6000539
Reviewed-by: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Alex Kravchuk <alexanderkr@google.com>
8 months ago
Struan Shrimpton b0024901d3 Add support for tagging bot_update step
In order to make reproduction steps appear on the bot_update step it
needs to have a tag that matches the instruction set on the invocation.

Bug: 349529661
Change-Id: Idcf75715ea4a64a5c6d9571f4f5485613864b7da
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5754258
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
12 months ago
Garrett Beaty 4750254802 Remove json and presentation attributes from bot_update.Result.
All downstream uses have been updated, so the backward compatibility is
no longer necessary.

Bug: 339472834
Change-Id: Ie8434168e7d1255561e5f5434ea811f753f88835
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5534148
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Garrett Beaty <gbeaty@google.com>
1 year ago
Garrett Beaty fd81abb19d Return a custom result type from bot_update.ensure_checkout.
In order to facilitate removing uses of api.path.checkout_dir from
downstream repos, this change adds a custom return type for
bot_update.ensure_checkout. Now instead of a standard step result, an
object of Result will be returned. Result records the relevant paths
(directory where the checkout was performed, the repo that was checked
out and the repo that was patched, if any). This provides the caller
the ability to work in any of these directories without using
api.path.checkout_dir and without requiring boilerplate to construct
the paths. It also includes some attributes that provide details from
within the json output to abstract that out.

Bug: 329113288, 339472834
Change-Id: I2ec6db635c5b799bdb65d4e9364e7d99aae4159e
Recipe-Manual-Change: build
Recipe-Manual-Change: build_limited
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5523194
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Garrett Beaty <gbeaty@google.com>
1 year ago
Garrett Beaty 7cc2678a05 Simplify overriding test data for bot_update.
Currently overriding test data for bot_update requires specifying the
json output and in some cases the corresponding retcode. It's easy to
set the json output to values that wouldn't match the build inputs. This
change adds module test data for bot_update to have the bot_update fail
to checkout or fail the patch without having to manually specify the
json output as well. It also adds module test data for controlling
whether there are commit positions in the output properties.

The fail_patch property of the bot_update module was removed since it
was only used by tests to force the patch failure codepath and now the
fail_patch module test data can be used instead.

Change-Id: I03f61c2ff95c2561b746efbaae8e35b520027a2b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518044
Reviewed-by: Scott Lee <ddoman@chromium.org>
Auto-Submit: Garrett Beaty <gbeaty@google.com>
Commit-Queue: Scott Lee <ddoman@chromium.org>
1 year ago
Rob Mohr 1dae848026 Replace use of "cq" module with "cv"
Bug: 333811087
Change-Id: I85b7d5c38defc03a580685654d43d204dbfa5231
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5454164
Auto-Submit: Rob Mohr <mohrr@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
1 year ago
Gavin Mak 642a989069 Reland "Drop py2 support in recipe modules"
This is a reland of commit 5819c303f0

Original change's description:
> Drop py2 support in recipe modules
>
> python3 is the only supported version of python in depot_tools.
>
> Bug: 1475402
> Change-Id: I479de09a0c34b438aced35e4ced58a5972108132
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4808518
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
> Commit-Queue: Gavin Mak <gavinmak@google.com>

Bug: 1475402
Change-Id: Ie4e6612a95511026b7602837f8dd3030bcf3ef8a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4811285
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2 years ago
Gavin Mak a5f17dd7b3 Revert "Drop py2 support in recipe modules"
This reverts commit 5819c303f0.

Reason for revert: breaking dart builds

Original change's description:
> Drop py2 support in recipe modules
>
> python3 is the only supported version of python in depot_tools.
>
> Bug: 1475402
> Change-Id: I479de09a0c34b438aced35e4ced58a5972108132
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4808518
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
> Commit-Queue: Gavin Mak <gavinmak@google.com>

Bug: 1475402
Change-Id: I0fef7581835bb25ea7e4f22303eef11e72df9536
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4811596
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Gavin Mak 5819c303f0 Drop py2 support in recipe modules
python3 is the only supported version of python in depot_tools.

Bug: 1475402
Change-Id: I479de09a0c34b438aced35e4ced58a5972108132
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4808518
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2 years ago
Gavin Mak 6fde53139b Upload git traces from bot_update
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Bug: 1351547
Change-Id: I1e018a8407c10ad4f9b6674d4b17ccae1e863389
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4568628
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2 years ago
Joanna Wang d04720454a Update recipes python tests.
For: http://shortn/_j5mixA1Mjr

Change-Id: I7ac4471fbb1e0629d59263266b3470e372d03a19
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4368938
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
2 years ago
Ravi Mistry f7e510b2f7 Add download-topics support to bot_update
Tested by:
* `python recipes.py test run`
* Tested end-to-end by patching in this unsubmitted change in Skia's recipes with https://skia-review.googlesource.com/c/skia/+/532768

Bug: chromium:1319415
Change-Id: Ia1c9c495ef6482b3fdb494e1c1c9320541bcd0c5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3602901
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Ravi Mistry <rmistry@chromium.org>
3 years ago
Gavin Mak d6d5c46cf4 Add PYTHON_VERSION_COMPATIBILITY to gitiles and bot_update
Setting PYTHON_VERSION_COMPATIBILITY = PY2+3 makes "recipe.py test" run
tests in both py2 and py3.

Bug: 1227140
Change-Id: I56a47f0491cbade172e1dc1239cbf6c2a0faa64f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3116850
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
4 years ago
Stephanie Kim da6894f7a9 Add enforce_fetch to ensure_checkout cmd arg
The enforce_fetch kwarg was being ignored. This change adds the
--enforce_fetch to the resources/bot_update cmd.
Confirmed here through led: b852fa5e4bb77d6fb2aa69ec87ee57b5d2/+/steps/bot_update/0/logs/execution_details/0

Bug: 1129235
Change-Id: I78c5d62841fda6dafbb709301e26c5047df2d58d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2444114
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Andrii Shyshkalov ef7a33909e bot_update: tell CQ to not retry patch application failures.
R=erikchen

Recipe-Nontrivial-Roll: build
Change-Id: I171daa3d61b2c20f963b8d587df8662740d80e71
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2050537
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Erik Chen <erikchen@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Andrii Shyshkalov <tandrii@google.com>
5 years ago
danakj c41f72cf44 Allow the bot_update api to not fetch tags from the git server.
There are 3 layers modified, starting from the bottom up:

1. git_cache.py populate

Now takes a --no-fetch-tags option. If specified, the cache will not
fetch updated tags from the server by passing --no-tags to git fetch.
This prevents the server from sending all the tag refs. In chromium
this prevents significant time bottlenecks dealing with 10k+ tags.

2. bot_update.py

bot_update has to deal with multiple git repos, it has the root repo
that is checked out through git-cache, as well as repos synched via
DEPS and gclient.

The script now takes a --no_fetch_tags option. If specified,
the git-cache populate call will include --no-fetch-tags. Otherwise, it
won't. So this controls (for chromium) if fetches to the src.git server
are passed --no-tags.

3. bot_update/api.py

This is the entry point for recipes and also has to deal with multiple
git repos. The behaviour at this point is not changed from the default.
A |no_fetch_tags| parameter is added to ensure_checkout() that defaults
to False.


This CL is a refactor with no intended behavior change.

The next step will be to change the chromium build repo to pass True
for ensure_checkout(no_fetch_tags) on chromium trybots.

This represents solution #2 in
https://docs.google.com/document/d/1hDIunJjjfpmr50y3YnZ4o3aq1NZF4gJa1TS0p7AIL64/edit#

Bug: 1019824
Change-Id: I935430603299daa9e301a95a5184c0ce486fd912
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1894352
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Erik Chen <erikchen@chromium.org>
Commit-Queue: danakj <danakj@chromium.org>
Auto-Submit: danakj <danakj@chromium.org>
6 years ago
Sergiy Belozorov 151c3484da Replace post-process checks with ones that are not deprecated
R=iannucci@chromium.org

Bug: 899266
Change-Id: Ia9b1f38590d636fa2858a2bd0bbf75d6b2cfe8fa
Reviewed-on: https://chromium-review.googlesource.com/c/1483033
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
6 years ago
John Budorick 1333933200 bot_update: stop failing with a KeyError in the recipe on failure.
This has been low-key driving me bananas for a while, and I've seen
it confuse sheriffs etc many times.

Change-Id: If46ab55821447a3a70a3f0ea774c017f29fc7400
Reviewed-on: https://chromium-review.googlesource.com/c/1286819
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
7 years ago