This is a reland of commit 5819c303f0
Original change's description:
> Drop py2 support in recipe modules
>
> python3 is the only supported version of python in depot_tools.
>
> Bug: 1475402
> Change-Id: I479de09a0c34b438aced35e4ced58a5972108132
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4808518
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
> Commit-Queue: Gavin Mak <gavinmak@google.com>
Bug: 1475402
Change-Id: Ie4e6612a95511026b7602837f8dd3030bcf3ef8a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4811285
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
python3 is the only supported version of python in depot_tools.
Bug: 1475402
Change-Id: I479de09a0c34b438aced35e4ced58a5972108132
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4808518
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
By default during tests, the _get_footers_step call will return None for
the returned footers because it does not set any test data. This does
not match the actual behavior of the recipe because git_footers.py
returns an empty json object if the commit message has no footers. This
change updates it to return an empty dict instead of None and checks for
the cached footers value to be None instead of any non-true value so
that a commit message without footers doesn't get parsed multiple times.
Change-Id: I716a27e964eb92de138228df5cc0876322d82823
Recipe-Manual-Change: build
Recipe-Nontrivial-Roll: build_limited
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4492269
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
https://crrev.com/c/4420526 put a 10s timeout on all calls made via
this script.
But it appears there are a variety of endpoints that take more than
10s, the latest being "SubmitChange", eg:
https://ci.chromium.org/ui/p/chromeos/builders/release/release-R113-15393.B-orchestrator/25/overview
So instead of applying the timeout to everything, and then opting
certain long-running calls out of it, this just opts-in the
"GetChanges" call to the timeout. This should cover the failure mode
that was originally solved for chrome's bots. And also bump the
timeout to 30s since we don't trust all get-changes queries to
reliably finish in under 10s.
This also bumps the step level timeout for the query in tryserver
recipe_mod to 8 min since there could be ~5min worth of sleeping +
150s worth of waiting+timing-out. So worst case the step will now
take 8min to fully exhaust all timeouts/sleeps.
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Bug: b/278083716
Change-Id: Ib366e004e0bb07297ba732590d488cae779e38ba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4426524
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
When gerrit's availability drops, chrome's builds see an excessive
amount of failures, eg:
https://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359780/overviewhttps://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359594/overviewhttps://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359564/overviewhttps://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359630/overview
Seemingly all failures occur in either the `gerrit fetch current CL
info` step or the `gerrit changes` step. Both steps have a 60s timeout
and shell out to depot_tools' gerrit_client.py. That script essentially
makes a single http call to gerrit. That request has no configured
timeout. So when gerrit's MIA and the call hangs indefinitely, so too
will the step hang. 60s after that, the step timeout is reached, and the
entire build crashes with an infra-failure.
However, one single retry has been shown to sufficiently work around
at least one instance of that failure:
dea9a6eca26f690ce10a959701a8fb51ad/+/build.proto
So this incorporates timeouts into the requests made by
gerrit_util.py. Each request is given a 10s timeout, which should be
enough for most/all gerrit calls. (Both steps have a p90 of less than
1sec.) When a timeout is reached, the script will automatically retry
up to 4 times.
This also bumps the timeouts of the step calls to gerrit_client.py to
6min since the script can now take up to 5 minutes to fail, ie:
the sequence of sleeps is roughly 10s, 20s, 40s, 80s, 160s, which is
about 5min. So a 6min timeout should cover all those retries.
This also passes in "--verbose" to all step calls to this script, so
the logging that prints out the retry + sleep log lines will be
visible in build logs.
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Bug: 1432638
Change-Id: I9dc47f4beeda3783ae4f9152bd29ee441ac3e197
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4420526
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
The python module is deprecated in favor of using the step module with
an explicit python binary. Switching the code to use the step module and
remove DEPS on recipe_engine/python removes warnings when running recipe
tests in the repo.
Change-Id: If3c9d5d6aff5aab69898b535399accbef92f5c0b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3880624
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.
The build that created this CL was
https://ci.chromium.org/b/8815259687168433617
Please review the expectation changes, and LGTM+CQ.
recipe_engine:
4cb1bf0fb0
4cb1bf0 (gbeaty@chromium.org)
Don't set the -u flag when running python unbuffered.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
R=iannucci@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Ignore-Freeze: Autoroller
Bugdroid-Send-Email: False
Change-Id: I0859db0f9db1c7548068073300150e979ac460b4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3621157
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Auto-Submit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Reviewed-by: Garrett Beaty <gbeaty@google.com>
The get_files_affected_by_patch method on the test API will return a
TestData object that determines the output of the "git diff" call. a
Test was added specifically for the get_files_affected_by_patch API
method to ensure that the test API operates as expected.
Change-Id: Ia1846700461757a72ed518894a5a38a8ca54fcc9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3422006
Auto-Submit: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Garrett Beaty <gbeaty@google.com>
This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.
The build that created this CL was
https://ci.chromium.org/b/8835439534712066113
Please review the expectation changes, and LGTM+CQ.
recipe_engine:
63c685b872~..f28ac0ab20025138a3f46845de09d625960c2a97
63c685b (chanli@chromium.org)
[buildbucket] test_api: emulate luci_context based on build info
f28ac0a (chanli@chromium.org)
Reland "[recipe-py] expose resultdb info in LUCI_CONTEXT"
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
R=nodir@chromium.org, tandrii@google.com
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: If2d4beb72d786f852feca9f4d43e08778bf71f3b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3174366
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
Auto-Submit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Allows for quick retrieval of host and project without having to parse
gerrit_change_repo_url or hard coding.
Bug:
Change-Id: I43f6e0bfda4b54d742bfa7ea20ee781162796608
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2971110
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
In crbug.com/1196159, a build got stuck at "gerrit changes" step for 3
hours. It would be good to add a timeout so that it fails early.
Also adjust the timeout for get_changes to 60s, from 600s
Bug: 1196159
Change-Id: Id2327ab40dd2fd8d95aaad6bb724f7ed06e6c244
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: build
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2811899
Commit-Queue: Haiyang Pan <hypan@google.com>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
To enable aggregate analysis on changed files in BigQuery, conditionally
report them via an output property.
This CL causes a non-trivial roll only because of the sorting.
Recipe-Nontrivial-Roll: build
Bug: 1151655
Change-Id: Ie7a6c622196143c1aef07af9a9a356fb4202ea56
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2553349
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Currently bot_update and tryserver crashes at initialization step if
the input contains multiple gerrit changes. This is not friendly for
CrOS CLs, e.g. crrev.com/c/2536761.
This CL moves the “crash” out of initialization, thus the consumer
of these modules could control which gerrit change to pass to bot_update
and tryserver.
This CL does NOT enable the modules here to support multiple patchset,
but allows consumers to choose one.
Bug:1146487
Change-Id: Iec46a8041189912e34da9bc5e8ff4611123ec9e0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528670
Commit-Queue: Xinan Lin <linxinan@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
This is a reland of e1410883a3
Use vpython to execute git_cl.py, gerrit_util.py and presubmit_support.py on recipes.
Original change's description:
> depot_tools: Add httplib2 to .vpython
>
> Check that things won't break before further changes are made.
>
> Bug: 1002153
> Change-Id: I41866f26334bf9ec2732bc0f25007234a95130e4
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1854749
> Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Bug: 1002153
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: release_scripts
Change-Id: Id94057eae8830dec197257df3ea35c0c4ff946b7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1856650
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
To sanely support concurrency, I need to make `api.step.nest` strictly
scoped; i.e. once you leave the nest context it closes all the steps
within that context.
This changes some wild 'active_step' uses to generate a step for the
module's own purpose.
R=martiniss@chromium.org, nodir@chromium.org, tandrii@chromium.org
Recipe-Nontrivial-Roll: build
Bug: 910369
Change-Id: I19aef31118faf9d3d078a2c5a0ebd48a4bfebd43
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1630044
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
The function can be called by chromium_test to set a property 'do_not_retry'
which will be propagated into buildbucket output.
Change-Id: I32d8ea925b7cb98d9b25d24226686e116c17801c
Bug: 910193
Reviewed-on: https://chromium-review.googlesource.com/c/1351542
Commit-Queue: Erik Chen <erikchen@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Ben Pastene <bpastene@chromium.org>
It is not used. get_gerrit_patch_root is used instead.
Bug: 694348
Change-Id: Icc717e38cbcaa8050c8ce8a364202b81d7f9cf2c
Reviewed-on: https://chromium-review.googlesource.com/1231282
Auto-Submit: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Rob Mohr
Gavin Mak
recipe-roller
Garrett Beaty
Ariel Zhang
Ben Pastene
Takuto Ikuta
Xinan Lin
Andrew Grieve
Josip Sokcevic
Stephanie Kim
Haiyang Pan
Greg Guterman
Nodir Turakulov
Xinan Lin
Andrii Shyshkalov
Andrii Shyshkalov
Edward Lemur
Michael Achenbach
Erik Staab
John Budorick
Robert Iannucci
erikchen
Stephen Martinis
Nodir Turakulov