This reverts commit 4cebc13a4e.
Reason for revert: this breaks Dart's builds
https://ci.chromium.org/ui/p/dart/builders/ci.sandbox/dart2js-unit-linux-x64-release/27481/infra
Original change's description:
> bot_update: remove cleanup_dir
>
> when git cache is removed, gclient sync fails
> and old checkout is moved into cleanup_dir.
>
> It will cause out of disk error during build
> because cleanup_dir keeps full old checkout and build artifacts,
> which is no longer needed when bot_update
> succeeded to checkout.
>
> Remove cleanup_dir after checkout.
>
> Bug: 388307198
> Change-Id: I3858e718622a1ef2012fe5b80a5d9b128d96d1d4
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6155955
> Commit-Queue: Jie Sheng <jiesheng@google.com>
> Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
> Auto-Submit: Fumitoshi Ukai <ukai@google.com>
> Owners-Override: Jie Sheng <jiesheng@google.com>
> Reviewed-by: Junji Watanabe <jwata@google.com>
> Reviewed-by: Jie Sheng <jiesheng@google.com>
Bug: 388307198
Change-Id: Ifb68c1aaf28e2203ba2c3b5804c2fd4939bbd921
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6159604
Auto-Submit: Alexander Thomas <athom@google.com>
Reviewed-by: Jie Sheng <jiesheng@google.com>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
when git cache is removed, gclient sync fails
and old checkout is moved into cleanup_dir.
It will cause out of disk error during build
because cleanup_dir keeps full old checkout and build artifacts,
which is no longer needed when bot_update
succeeded to checkout.
Remove cleanup_dir after checkout.
Bug: 388307198
Change-Id: I3858e718622a1ef2012fe5b80a5d9b128d96d1d4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6155955
Commit-Queue: Jie Sheng <jiesheng@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Owners-Override: Jie Sheng <jiesheng@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
Reviewed-by: Jie Sheng <jiesheng@google.com>
This function was a shim to ensure consistency between Python 2 and 3.
It's no longer necessary now that recipes don't support Python 2
anymore, and is being deleted in crrev.com/c/6133284.
Also update two tests to be less sensitive to the exact format of the
summary markdown produced when an exception occurs, since some single
quotes are now changed to double quotes.
Change-Id: Ia8a7bd9be5f03302049f6798c51b8e4e8bdc9784
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6142074
Reviewed-by: Gavin Mak <gavinmak@google.com>
Auto-Submit: Oliver Newman <olivernewman@google.com>
Commit-Queue: Oliver Newman <olivernewman@google.com>
The gclient config refers to a repo that is no longer used and includes
non-inclusive language.
Bug: 40712760
Change-Id: Id7faec4beefdc4387cd8860910e43cbdcac3625a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6070412
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
crrev.com/c/6049353 updated bot_update.ensure_checkout to comput the
output commit even when not setting the output commit so that callers
could do custom logic for setting output commits while still having
access to the one computed by bot_update. Unfortunately, this changed
behavior for some builds that have set_output_commit set false. When
computing the output commit, if none of the conditions that set the ref
on the output commit were true then an AssertionError would be raised
indicating that the caller should call
buildbucket.set_output_gitiles_commit. With crrev.com/c/6049353, the
assertion can be raised even if set_output_commit is false.
This changes it so that the constraint on the ref field is only enforced
when set_output_commit is true.
Bug: 366409421
Change-Id: Ifb3a39784d75a3c47a7d4a6e2ac9957c29a09938
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6064719
Auto-Submit: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
out_commit is parsed from got_revision even when set_output_commit is False. This allows for the calling code to implement its own logic for setting output commit, but still use the commit from bot_update if needed.
Example of usage: https://crrev.com/c/6049557.
Bug: 366409421
Change-Id: I49ce89730fe780d7561077f82797b30fad87e65e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6049353
Reviewed-by: Garrett Beaty <gbeaty@google.com>
Commit-Queue: Alex Kravchuk <alexanderkr@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
This reverts commit 66b3972fc5.
Reason for revert: This won't work because chromium_checkout relies on the position being set in the output commit in order to set information for RDB. We'll have to come up with a different solution.
Original change's description:
> Add parse_commit_position parameter to BotUpdateApi.ensure_checkout.
>
> When true and got_revision_cp is present, output commit ref and position are set from got_revision_cp. Default value is true.
>
> For Chrome builds that build from tags we will set parse_commit_position to false to show tags instead of branches on Buildbucket UI, e.g. refs/tags/132.0.6824.0 instead of refs/branch-heads/6824@{#1}.
>
> Bug: 366409421
> Change-Id: I77d01615edb6b791445a06469f80c673c97ad8d6
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6000539
> Reviewed-by: Garrett Beaty <gbeaty@google.com>
> Reviewed-by: Scott Lee <ddoman@chromium.org>
> Commit-Queue: Alex Kravchuk <alexanderkr@google.com>
Bug: 366409421
Change-Id: I21a5fc6e011a64fffd1d2ef0689d5a3f3843482c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6037603
Reviewed-by: Garrett Beaty <gbeaty@google.com>
Commit-Queue: Alex Kravchuk <alexanderkr@google.com>
Reviewed-by: Scott Lee <ddoman@chromium.org>
Rename parse_commit_position parameter to parse_commit_position_for_tags. If input commit ref is not a tag, or parse_commit_position_for_tags is True, execute the old code path. This makes the changes backward compatible with the existing code.
Bug: 366409421
Change-Id: I58405325e406a82c2e255fe5d3d4a2883c98a84a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6035030
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Alex Kravchuk <alexanderkr@google.com>
Reviewed-by: Garrett Beaty <gbeaty@google.com>
The flag to filter for branches was renamed in git 2.46.0 [1] from
`-h` to `-b`. This causes previous versions of git to fail when running
ls-remote.
To be compatible across a version upgrade, a second git call would be
required to retrieve the version and determine the correct flag.
However, the filter feature is not used yet and can also be solved by
specifying the absolute ref path within the recipe. Therefore, this
change removes ls-remote's filter flags.
[1] https://git-scm.com/docs/git-ls-remote/2.42.0
R=machenbach@chromium.org, gavinmak@google.com
Bug: none
Change-Id: I840fb3ac56daf47c798d5c32f00de841b8db2771
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6028418
Auto-Submit: Alexander Schulze <alexschulze@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Call `git ls-remote` to determine the head revision for a branch, or
the revision for a tag. The method is useful for bots which do not
require a checkout.
Bug: none
Change-Id: Ibc01d1fb9c2714bd4d719159dbdaf3d7684ea1e9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6012563
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Alexander Schulze <alexschulze@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
When true and got_revision_cp is present, output commit ref and position are set from got_revision_cp. Default value is true.
For Chrome builds that build from tags we will set parse_commit_position to false to show tags instead of branches on Buildbucket UI, e.g. refs/tags/132.0.6824.0 instead of refs/branch-heads/6824@{#1}.
Bug: 366409421
Change-Id: I77d01615edb6b791445a06469f80c673c97ad8d6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6000539
Reviewed-by: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Alex Kravchuk <alexanderkr@google.com>
To tag reproduction steps on chromium update we need to generate a tag
for the step. Rather than duplicate this piece of code, expose it
in the bot_update module
Bug: 349529661
Change-Id: Ibed143523b3b1af4de222a8617349b815f0c0aaa
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5914637
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
In order to make reproduction steps appear on the bot_update step it
needs to have a tag that matches the instruction set on the invocation.
Bug: 349529661
Change-Id: Idcf75715ea4a64a5c6d9571f4f5485613864b7da
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5754258
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
Tested by running `vpython3 gerrit_client.py` so at least everything
loads with 3.11
Change-Id: I925fce1ce3fd4c427a57fb590c991bf49b99c1be
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5741852
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Allen Li <ayatane@chromium.org>
Buildspecs haven't been used since 2021. Downstream code that sets the
BUILDSPEC_VERSION field has been removed.
Bug: 353561017
Change-Id: Iaf88ec73bed6b371a97fbdb6588a74b34e3b87fc
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5718628
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Garrett Beaty <gbeaty@google.com>
bot_update sets disk usage before checkout in step_text.
This CL adds disk usage after checkout in step_text.
Bug: 351868766
Change-Id: I2d99c8a3e0bd8829df30d349789ac8be7086ccf4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5717616
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
This reverts commit 1c0a575c4c.
Reason for revert: Suspected of causing infra failures on
bots like https://ci.chromium.org/ui/p/dawn/builders/try/cmake-linux-clang-dbg-x64/b8742144151501823569/infra.
Long-term should be fixed by removing the invalid configs,
but need to prevent these infra failures for now.
Original change's description:
> Add a recipe property to override bot_update's STALE_PROCESS_DURATION.
>
> Bug: 353444802
> Change-Id: I8d174e28fbbd552be591c8a4c1308a29de47d0b6
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5713191
> Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
> Commit-Queue: Brian Ryner <bryner@google.com>
Bug: 353444802
Change-Id: I0a109304c6de16f96ceff7ca1e64e3cd10d6cf4a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5718634
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Brian Ryner <bryner@google.com>
All downstream uses have been updated, so the backward compatibility is
no longer necessary.
Bug: 339472834
Change-Id: Ie8434168e7d1255561e5f5434ea811f753f88835
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5534148
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Garrett Beaty <gbeaty@google.com>
In order to facilitate removing uses of api.path.checkout_dir from
downstream repos, this change adds a custom return type for
bot_update.ensure_checkout. Now instead of a standard step result, an
object of Result will be returned. Result records the relevant paths
(directory where the checkout was performed, the repo that was checked
out and the repo that was patched, if any). This provides the caller
the ability to work in any of these directories without using
api.path.checkout_dir and without requiring boilerplate to construct
the paths. It also includes some attributes that provide details from
within the json output to abstract that out.
Bug: 329113288, 339472834
Change-Id: I2ec6db635c5b799bdb65d4e9364e7d99aae4159e
Recipe-Manual-Change: build
Recipe-Manual-Change: build_limited
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5523194
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Currently overriding test data for bot_update requires specifying the
json output and in some cases the corresponding retcode. It's easy to
set the json output to values that wouldn't match the build inputs. This
change adds module test data for bot_update to have the bot_update fail
to checkout or fail the patch without having to manually specify the
json output as well. It also adds module test data for controlling
whether there are commit positions in the output properties.
The fail_patch property of the bot_update module was removed since it
was only used by tests to force the patch failure codepath and now the
fail_patch module test data can be used instead.
Change-Id: I03f61c2ff95c2561b746efbaae8e35b520027a2b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518044
Reviewed-by: Scott Lee <ddoman@chromium.org>
Auto-Submit: Garrett Beaty <gbeaty@google.com>
Commit-Queue: Scott Lee <ddoman@chromium.org>
The downstream use has been switched to set patch_root instead so the
root parameter is no longer necessary.
Change-Id: Id0b20265550daa7e93af0ebf717d48db56164037
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5514209
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
In production, if there is no patch, then the bot_update output will
have the value for the patch_root key set to null and the patch_failure
key will only be set if there was a patch failure. This updates the
output_json test API method so that the json matches this behavior. The patch_root parameter was added that will be used to set the
patch_root key. The root parameter is preserved for now until a
downstream use passing the argument by keyword can be updated.
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Change-Id: I0c34a41ff18bce2900a2f176ff992e6d147abbd4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5506838
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Replace uses of '[START_DIR]' and '[CACHE]' in comments with
api.path.start_dir and api.path.cache_dir. This doesn't actually keep
anything from breaking, but it ensures the comments stay correct.
Change-Id: I4087925a29e3e9a4cb47e044733ab6109cb659bd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5492788
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Rob Mohr <mohrr@google.com>
Use Path.joinpath() or the / operator instead of Path.join().
Eventually, config_types.Path will be replaced with pathlib.Path, which
doesn't have a join() method. In general, if the substitution was simple
the / operator was used. Otherwise, '.join(' was replaced with
'.joinpath('.
Change-Id: Iff2eb0abd9fdb0d3b3bca281c7c5436f932a5380
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5483206
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Rob Mohr <mohrr@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
config_types.Path will soon be replaced with pathlib.Path and
pathlib.Path doesn't have a join() method.
Bug: 329113288
Change-Id: I81bf607ffb41d84928af0eea06be78540b9912d9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5467697
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Rob Mohr <mohrr@google.com>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
KS Yim
Dirk Pranke
Mike Frysinger
Leonard Grey
Fumitoshi Ukai
Vadim Shtayura
Jie Sheng
Alexander Thomas
recipe-roller
Oliver Newman
Garrett Beaty
Alex Kravchuk
Ben Pastene
Congbin Guo
Alex Schulze
Yiwei Zhang
Struan Shrimpton
Josip Sokcevic
Allen Li
Brian Ryner
Brian Sheedy
Rob Mohr