python3 is the only supported version of python in depot_tools.
Bug: 1475402
Change-Id: I479de09a0c34b438aced35e4ced58a5972108132
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4808518
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
The step_result json can currently be empty if the orchestrator build
is canceled while running bot_update. Trying to do
step_result.json.output in the finally block raises an "Uncaught
exception AttributeError".
Ex: https://ci.chromium.org/ui/p/chromium/builders/try/win10_chromium_x64_rel_ng/1121568/overview
This change checks for the json attr before trying to read json.output.
Bug: 1305332
Change-Id: I1dc277642ece1b96144be6381a7c4b8426cb2bc6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3518372
Reviewed-by: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
gclient removed syntax validation flag in 2020:
https://crrev.com/c/2083589, so this can safely be removed.
R=aravindvasudev@google.com, gavinmak@google.com
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: build
Change-Id: I2aced6cdb01d571ca361cc57e7c8715c9ca40552
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3419278
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
* If the input revision starts with refs/ use it as the ref of the CP.
* If the input revision is of the form origin/X, use refs/heads/X as the
ref of the CP.
Change-Id: I048883afcfe1593e877c004729f859921910c993
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3165297
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Auto-Submit: Garrett Beaty <gbeaty@google.com>
Setting PYTHON_VERSION_COMPATIBILITY = PY2+3 makes "recipe.py test" run
tests in both py2 and py3.
Bug: 1227140
Change-Id: I56a47f0491cbade172e1dc1239cbf6c2a0faa64f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3116850
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
When set_output_commit is True and the repo associated with the
got_revision property does not use commit positions, buiders whose
gclient config uses RevisionFallbackChain for that repo will raise an
exception when attempting set the output commit because a string is
expected for the revision value, not a RevisionFallbackChain object.
This changes the set_output_commit code to resolve the revision so that
the output commit can be set in those cases.
Bug: 1159749, 1141886
Change-Id: Iddf1d49871028d9500eefb0eed07e36ce7e6b0d6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2612223
Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Currently, if the bot_update script fails, then recipe code that makes
assumptions about the returned output JSON cause exceptions to be
raised. This fixes a couple instances in the code so that the bot_update
failure should surface instead of an uncaught exception.
Change-Id: I031e613841b03e5eb4f6d9ca7adb5085088a3077
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2405708
Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Right now the default test data for bot_update doesn't transform
revisions specifying refs for the revision except for refs starting with
refs/ on the first solution. This means that the output will just have
the ref name as the revision value, which isn't compatible with code
that requires an actual revision value.
Bug: 1047893
Change-Id: I016ad5e8db5a293dd5e6dacb5b8beba51e000949
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: chrome_release
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2325761
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
This adds a parameter to bot_update.ensure_checkout that will cause
blamelist pins to be set for all of the repos whose revision is not
determined by DEPS files: the repo for the gitiles input, the repo for
each solution in the gclient config and any repos with the revision set
by the revisions field of the gclient config.
Bug: 1047893
Change-Id: Ie3eee41dbf15be014185b6f255ee3d4cfd40b4e4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2293514
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Currently the output_json method generates revisions for projects that
are in the revision_mapping and only those projects appear in the
manifest and source_manifest, using the generated revision. This method
is used to construct the default json output for bot_update in tests.
This has 2 problems:
1. If a project has a fixed revision, its manifest entries and the
property used the generated revision, meaning the gitiles input revision
will not be used in the json output.
2. If a project has a fixed revision and isn't present in
revision_mapping then manifest entries aren't generated for it.
This changes the method to use the revisions in the fixed_revisions dict
and to create manifest entries for the projects that appear in the
fixed_revisions dict which should make fewer situations where
bot_update's output_json needs to be explicitly set.
Change-Id: I99de52edc9248c28e9d3aa3a04cc77bf9fccbe37
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: chrome_release
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2304644
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
This is essentially defunct; We will want to add something like this
back in, but it's so sparsely used right now that it's better to just
remove it and then add it back properly when we're ready to support
it across all of LUCI.
R=yiwzhang@google.com
Bug: 1093959
Change-Id: I72673515d860234daa3b01514ac38168ebb4825a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2250403
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
We no longer need it to bridge path differences between buildbot and
LUCI, and it shouldn't be in use downstream now either.
R=machenbach@chromium.org, martiniss@chromium.org, tandrii@chromium.org, yiwzhang@google.com
Bug: 1015181
Change-Id: Iaba2100653ae9f06799d1d510bef993f7f710dc4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2220171
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
This reverts commit 512f92b731.
Reason for revert: bot_update is having infra failure on ci bots.
For example https://ci.chromium.org/p/chromium/builders/ci/Closure%20Compilation%20Linux/249684.
Original change's description:
> Fetch the input commit ref when both ref and id are specified.
>
> This causes --refs refs/head/master to be added to bot_update
> calls when the input build message contains a commit id and the
> refs/head/master ref. This is an effective no-op.
>
> Bug: 922150
> Change-Id: I119d9b3cede727713a0b66bff83b5ee4e1848790
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1884457
> Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
> Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
TBR=tandrii@google.com,gbeaty@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 922150
Change-Id: Ie9275c21827688f59449f052c768452a8652e139
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1891000
Commit-Queue: Haiyang Pan <hypan@google.com>
Reviewed-by: Haiyang Pan <hypan@google.com>
This causes --refs refs/head/master to be added to bot_update
calls when the input build message contains a commit id and the
refs/head/master ref. This is an effective no-op.
Bug: 922150
Change-Id: I119d9b3cede727713a0b66bff83b5ee4e1848790
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1884457
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
{_resolve_fixed_revisions} resolves every revision property. After a test
failure, we don't want to resolve every revision always.
This CL allows updating a subset of revisions for a new checkout.
R=machenbach@chromium.orgTBR=tandrii@chromium.org
Bug: 950731, 982711
Change-Id: I3cdc5cdb882009886b82557427fdf9b02e860b39
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1720929
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Auto-Submit: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
got_revision and got_revision_cp point to the commit that
Build.output.gitiles_commit should point to.
Therefore, derive output commit from these properties.
Derive output ref from got_revision_cp if it available.
Otherwise, try to read it from input commit
Bug: 940214
Change-Id: I108d782ad93c70114581830d7427b95d3efd9d98
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1588506
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Add set_output_commit param to bot_update.ensure_checkout, false by default.
If true, call api.buildbucket.set_output_gitiles_commit to set the output commit
of the build. This generally supersedes got_revision and got_revision_cp.
Bug: 940214
Change-Id: I5ffc790a6b12c8bf4a56469b3ecc567141c0d735
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1548256
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
It has been True by default for a while, and there is no need to override it.
Bug: 891917
Change-Id: I2598a2230b0ea38a647a533757331c541b871971
Reviewed-on: https://chromium-review.googlesource.com/c/1260057
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Add tryserver.api.gerrit_change_info that returns detailed info about the
current Gerrit CL. It makes a Gerrit RPC in the background on the first read.
This step will be present in all tryjobs.
This mechanism can be used for extra details about the current CL.
This CL starts to use this in bot_update for gerrit_ref, and stops using
patch_ref property, and also update bot_update to use when the destination
branch is needed.
Also this CL removes no longer used properties in bot_update,
namely most of patch_* properties.
build.git manual roll: https://chromium-review.googlesource.com/c/chromium/tools/build/+/1217626
Recipe-Manual-Change: build
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: skiabuildbot
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: release_scripts
R=tandrii@chromium.org
Bug: 694348
Change-Id: I945d061b74278c3d1d227578abe16a64cdaca647
Reviewed-on: https://chromium-review.googlesource.com/1213557
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Use repo_path_map in all places where patch_projects is currently used.
Add gclient.get_repo_path function to resolve path to a repo given its URL.
infra.git requires a non-trivial roll because of incorrect expectations.
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Bug: 877161
Change-Id: Iaea0a3f14609bf0c18d4ecdf8564153b8052584d
Reviewed-on: https://chromium-review.googlesource.com/1208363
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
shallow mode is a special git mode that only checks out a shallow copy
of git so the full history is not downloaded. It is not well supported
by googlesource.com servers because git has to do a giant negotiation
with the server and transmit every single commit that the local checkout
has or doesn't have (instead of just the latest commit of each ref),
which ends up taking more than 10x longer than a non-shallow checkout.
R=ehmaldonado, hinoka, iannucci
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Bug: 855137
Change-Id: I5f9e31f8b2730cf80b79bfd0e08201c33eb942f5
Reviewed-on: https://chromium-review.googlesource.com/1110450
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
This is a reland of 3806b7fbd0
Original change's description:
> bot_update: Fetch references to be synced.
>
> If a solution is to be synced to a reference, such as
> refs/branch-heads/#### fetch it so that it can later be
> checked out.
>
> I tried this change on a LUCI task that runs chromium_presubmit
> for crrev.com/c/1048868. The bot_update works as expected:
> https://ci.chromium.org/swarming/task/3d5b294cff467710?server=chromium-swarm.appspot.com
>
>
> Bug: 840445
>
> Change-Id: I6444c0feb679ca89b2f4065215de915494e5082a
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Reviewed-on: https://chromium-review.googlesource.com/1049007
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Bug: 840445
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Change-Id: I64a97aafe5fab9a26c478262e9ca104dc59e61f5
Reviewed-on: https://chromium-review.googlesource.com/1052387
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
This reverts commit 3806b7fbd0.
Reason for revert: Broke downstream tot bots
Original change's description:
> bot_update: Fetch references to be synced.
>
> If a solution is to be synced to a reference, such as
> refs/branch-heads/#### fetch it so that it can later be
> checked out.
>
> I tried this change on a LUCI task that runs chromium_presubmit
> for crrev.com/c/1048868. The bot_update works as expected:
> https://ci.chromium.org/swarming/task/3d5b294cff467710?server=chromium-swarm.appspot.com
>
>
> Bug: 840445
>
> Change-Id: I6444c0feb679ca89b2f4065215de915494e5082a
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Reviewed-on: https://chromium-review.googlesource.com/1049007
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
TBR=mmoss@chromium.org,machenbach@chromium.org,tandrii@chromium.org,ehmaldonado@chromium.org
Change-Id: If94df8684e634504d8a0c11b2aa5862a458ba19f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 840445, 841306
Reviewed-on: https://chromium-review.googlesource.com/1052187
Reviewed-by: agrieve <agrieve@chromium.org>
Commit-Queue: agrieve <agrieve@chromium.org>
If a solution is to be synced to a reference, such as
refs/branch-heads/#### fetch it so that it can later be
checked out.
I tried this change on a LUCI task that runs chromium_presubmit
for crrev.com/c/1048868. The bot_update works as expected:
https://ci.chromium.org/swarming/task/3d5b294cff467710?server=chromium-swarm.appspot.com
Bug: 840445
Change-Id: I6444c0feb679ca89b2f4065215de915494e5082a
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Reviewed-on: https://chromium-review.googlesource.com/1049007
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Gavin Mak
Joanna Wang
Joanna Wang
Stephanie Kim
Josip Sokcevic
Garrett Beaty
Yiwei Zhang
Edward Lesmes
Takuto Ikuta
Robert Iannucci
John Budorick
Haiyang Pan
Tamer Tas
Nodir Turakulov
Edward Lemur
Nodir Turakulov
Andrii Shyshkalov
agrieve
Stephen Martinis