python3 is the only supported version of python in depot_tools.
Bug: 1475402
Change-Id: I479de09a0c34b438aced35e4ced58a5972108132
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4808518
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
By default during tests, the _get_footers_step call will return None for
the returned footers because it does not set any test data. This does
not match the actual behavior of the recipe because git_footers.py
returns an empty json object if the commit message has no footers. This
change updates it to return an empty dict instead of None and checks for
the cached footers value to be None instead of any non-true value so
that a commit message without footers doesn't get parsed multiple times.
Change-Id: I716a27e964eb92de138228df5cc0876322d82823
Recipe-Manual-Change: build
Recipe-Nontrivial-Roll: build_limited
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4492269
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
https://crrev.com/c/4420526 put a 10s timeout on all calls made via
this script.
But it appears there are a variety of endpoints that take more than
10s, the latest being "SubmitChange", eg:
https://ci.chromium.org/ui/p/chromeos/builders/release/release-R113-15393.B-orchestrator/25/overview
So instead of applying the timeout to everything, and then opting
certain long-running calls out of it, this just opts-in the
"GetChanges" call to the timeout. This should cover the failure mode
that was originally solved for chrome's bots. And also bump the
timeout to 30s since we don't trust all get-changes queries to
reliably finish in under 10s.
This also bumps the step level timeout for the query in tryserver
recipe_mod to 8 min since there could be ~5min worth of sleeping +
150s worth of waiting+timing-out. So worst case the step will now
take 8min to fully exhaust all timeouts/sleeps.
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Bug: b/278083716
Change-Id: Ib366e004e0bb07297ba732590d488cae779e38ba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4426524
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
When gerrit's availability drops, chrome's builds see an excessive
amount of failures, eg:
https://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359780/overviewhttps://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359594/overviewhttps://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359564/overviewhttps://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359630/overview
Seemingly all failures occur in either the `gerrit fetch current CL
info` step or the `gerrit changes` step. Both steps have a 60s timeout
and shell out to depot_tools' gerrit_client.py. That script essentially
makes a single http call to gerrit. That request has no configured
timeout. So when gerrit's MIA and the call hangs indefinitely, so too
will the step hang. 60s after that, the step timeout is reached, and the
entire build crashes with an infra-failure.
However, one single retry has been shown to sufficiently work around
at least one instance of that failure:
dea9a6eca26f690ce10a959701a8fb51ad/+/build.proto
So this incorporates timeouts into the requests made by
gerrit_util.py. Each request is given a 10s timeout, which should be
enough for most/all gerrit calls. (Both steps have a p90 of less than
1sec.) When a timeout is reached, the script will automatically retry
up to 4 times.
This also bumps the timeouts of the step calls to gerrit_client.py to
6min since the script can now take up to 5 minutes to fail, ie:
the sequence of sleeps is roughly 10s, 20s, 40s, 80s, 160s, which is
about 5min. So a 6min timeout should cover all those retries.
This also passes in "--verbose" to all step calls to this script, so
the logging that prints out the retry + sleep log lines will be
visible in build logs.
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Bug: 1432638
Change-Id: I9dc47f4beeda3783ae4f9152bd29ee441ac3e197
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4420526
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Removes a TODO to remove the patch_text argument and its uses from
get_footers and related methods. The TODO indicated that it was
only used in example code, and thus should be handled another way.
However, it turns out that there is at least one legitimate use of
thus functionality downstream to retrieve footers from CL
descriptions earlier revisions.
Fixed: 1179039
Change-Id: I79996f6053a6769043943de39735b2b5d7ee02ee
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4283342
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
The python module is deprecated in favor of using the step module with
an explicit python binary. Switching the code to use the step module and
remove DEPS on recipe_engine/python removes warnings when running recipe
tests in the repo.
Change-Id: If3c9d5d6aff5aab69898b535399accbef92f5c0b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3880624
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.
The build that created this CL was
https://ci.chromium.org/b/8815259687168433617
Please review the expectation changes, and LGTM+CQ.
recipe_engine:
4cb1bf0fb0
4cb1bf0 (gbeaty@chromium.org)
Don't set the -u flag when running python unbuffered.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
R=iannucci@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Ignore-Freeze: Autoroller
Bugdroid-Send-Email: False
Change-Id: I0859db0f9db1c7548068073300150e979ac460b4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3621157
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Auto-Submit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Reviewed-by: Garrett Beaty <gbeaty@google.com>
The get_files_affected_by_patch method on the test API will return a
TestData object that determines the output of the "git diff" call. a
Test was added specifically for the get_files_affected_by_patch API
method to ensure that the test API operates as expected.
Change-Id: Ia1846700461757a72ed518894a5a38a8ca54fcc9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3422006
Auto-Submit: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Garrett Beaty <gbeaty@google.com>
The tryserver.require_is_tryserver method will create an infra-failing
step that raises an exception if there is no CL associated with the
build. In the case of an LED task, it will only fail so as not to cause
the removal of the builder cache in the case of a minor user error.
Change-Id: I05d3a86cc727577f067fb3eae49e46398345f672
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3413971
Auto-Submit: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.
The build that created this CL was
https://ci.chromium.org/b/8835439534712066113
Please review the expectation changes, and LGTM+CQ.
recipe_engine:
63c685b872~..f28ac0ab20025138a3f46845de09d625960c2a97
63c685b (chanli@chromium.org)
[buildbucket] test_api: emulate luci_context based on build info
f28ac0a (chanli@chromium.org)
Reland "[recipe-py] expose resultdb info in LUCI_CONTEXT"
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
R=nodir@chromium.org, tandrii@google.com
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: If2d4beb72d786f852feca9f4d43e08778bf71f3b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3174366
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
Auto-Submit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Allows for quick retrieval of host and project without having to parse
gerrit_change_repo_url or hard coding.
Bug:
Change-Id: I43f6e0bfda4b54d742bfa7ea20ee781162796608
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2971110
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Gonna add a small warning in chromium's recipe when this footer is
used. Add it here so we can refer to the footer as a constant in
tryserver recipe_mod.
Bug: None
Change-Id: Ie4f9d1d208c0dad8d0df8f07b74e761da1051a1c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2864005
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
In crbug.com/1196159, a build got stuck at "gerrit changes" step for 3
hours. It would be good to add a timeout so that it fails early.
Also adjust the timeout for get_changes to 60s, from 600s
Bug: 1196159
Change-Id: Id2327ab40dd2fd8d95aaad6bb724f7ed06e6c244
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: build
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2811899
Commit-Queue: Haiyang Pan <hypan@google.com>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Includes all footers used by the recipe. Other footers used only by presubmit
hooks not included.
Bug: chromium:1079219
Change-Id: I9c351bda158bc7c1e64a8254feb7055dc31fb6db
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2707709
Auto-Submit: Jacob Kopczynski <jkop@chromium.org>
Commit-Queue: Jacob Kopczynski <jkop@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
It will be easier to support centralized management of what footers exist if we
require all callers to specifically declare which footers they want to check.
This is a partial step towards allowing that without increasing the number of
queries required. It introduced cache staleness as a potential issue, but should
not matter in practice.
Bug: chromium:1079219
Change-Id: I678fd366202bac5a4efa4258ffe02e70c64589bb
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: chromiumos
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2664790
Commit-Queue: Jacob Kopczynski <jkop@chromium.org>
Auto-Submit: Jacob Kopczynski <jkop@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
To enable aggregate analysis on changed files in BigQuery, conditionally
report them via an output property.
This CL causes a non-trivial roll only because of the sorting.
Recipe-Nontrivial-Roll: build
Bug: 1151655
Change-Id: Ie7a6c622196143c1aef07af9a9a356fb4202ea56
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2553349
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
This reverts commit 4447336100.
Reason for revert: Breaks infra repo. See crrev.com/c/2548285
Original change's description:
> Improve the setup logic for tryserver
>
> If the given change does not have a valid gerrit host name, stop
> initializing the tryserver module.
> Also polished the variable name and docstr added in crrev/c/2528670.
>
> Recipe-Nontrivial-Roll: infra
> Bug:1146487
> Change-Id: I38193b25d37c724fd41d898eb020237fbf7cecdf
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2544000
> Commit-Queue: Xinan Lin <linxinan@chromium.org>
> Reviewed-by: Nodir Turakulov <nodir@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
TBR=nodir@chromium.org,tandrii@google.com,recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com,linxinan@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com
Change-Id: I3871866454adab816ab5549f8273fe16af2d9adb
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1146487
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2548409
Reviewed-by: Xinan Lin <linxinan@chromium.org>
Commit-Queue: Xinan Lin <linxinan@chromium.org>
If the given change does not have a valid gerrit host name, stop
initializing the tryserver module.
Also polished the variable name and docstr added in crrev/c/2528670.
Recipe-Nontrivial-Roll: infra
Bug:1146487
Change-Id: I38193b25d37c724fd41d898eb020237fbf7cecdf
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2544000
Commit-Queue: Xinan Lin <linxinan@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Currently bot_update and tryserver crashes at initialization step if
the input contains multiple gerrit changes. This is not friendly for
CrOS CLs, e.g. crrev.com/c/2536761.
This CL moves the “crash” out of initialization, thus the consumer
of these modules could control which gerrit change to pass to bot_update
and tryserver.
This CL does NOT enable the modules here to support multiple patchset,
but allows consumers to choose one.
Bug:1146487
Change-Id: Iec46a8041189912e34da9bc5e8ff4611123ec9e0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528670
Commit-Queue: Xinan Lin <linxinan@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
This was made by running `codespell` and `scspell`
and then checking the results.
Change-Id: I169fd5b40294f83015075b4a899fbca263821f25
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2144602
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Auto-Submit: Quinten Yearsley <qyearsley@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Gavin Mak
recipe-roller
Robert Iannucci
Garrett Beaty
Ariel Zhang
Ben Pastene
Joanna Wang
Brian Sheedy
Takuto Ikuta
Xinan Lin
Andrew Grieve
Josip Sokcevic
Stephanie Kim
Haiyang Pan
Greg Guterman
Jacob Kopczynski
Nodir Turakulov
Xinan Lin
Quinten Yearsley
Andrii Shyshkalov
Andrii Shyshkalov
Edward Lemur
Michael Achenbach