python3 is the only supported version of python in depot_tools.
Bug: 1475402
Change-Id: I479de09a0c34b438aced35e4ced58a5972108132
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4808518
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
It's the only roll that uses use_relative_path = False. Instead of fully
supporting that case, we will ask user to provide prefix that should be
removed when gitlinks are added.
Change-Id: I1e39d0ce145a246e5b69b7cf64e6b9471bcb1aea
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4794721
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Keybo Qian <keybo@google.com>
roll_deps method in gclient recipe_module allow users to pass DEPS file
together with recipes that need to be rolled. It returns a list of files
that need to be edited in addition to the modified DEPS file (replaced
inline).
R=jojwang, keybo
Change-Id: I6987849f0ebacbb063d6f6186af1dba1647550a7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4785897
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Xinan Lin <linxinan@chromium.org>
The chromium-style plugin was replaced last year with cl/495068108.
Change-Id: Ifce7cf5e22f10681ffe743572182d759ae51fed2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4743665
Auto-Submit: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
The recipe engine has a refactor coming up where CONFIG_CTX needs to
be explicitly exported for other modules to extend it. Currently
a handful of modules in the build, build_internal and
release_scripts repos extend the gclient configuration context.
Exporting it like this should have no functional effect, but is just
in preparation for the upcoming refactor.
R=gavinmak
Bug: 1462728
Change-Id: I5871d472f7e28d38eefa9b1e89cfcc804823a1bd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4680284
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
This rolls adds support for xcode15 adoption, in case builders using osx recipes want to roll to xcode15 in the future
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Bug: 1457029
Change-Id: I958d1ac03281d5663ff3a0df4f0812bde863df2e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4675569
Reviewed-by: Jeff Yoon <jeffyoon@google.com>
Commit-Queue: Yue She <yueshe@google.com>
The usage of config in the git_cl module is wholly unnecessary - This
CL gives a trivial way to change downstream repos to use a normal method
in place of configs, which will ultimately let us remove config.py from
this module.
Change-Id: Ia02ad110c41fadcd3bada0258ba96cbe70d0e43c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4662591
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
The "upload traces" step is skipped if bot_update times out and is in
the grace_period. Reserve 1 minute to upload traces.
Bug: 1351547, 1456840
Change-Id: Idb191735a07d245575ff4c79375a976750522fa5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4651461
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Since python3.3, the platform value has been changed from linux2 or
linux3 to linux. The bug has stopped bot_update script from emitting
proces tree periodically that will be helpful when debugging.
R=sokcevic
Bug: 1455926
Change-Id: Id6ab4ddf46eaa322a8b5470baa7ba6cc656a6bb3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4654290
Auto-Submit: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
The blamelist_pins attribute is being set on the json output, which is a
dictionary. This only works right now because json.output currently uses
OrderedDict as the dict type. With removal of support for python2, dict
could be used since its iteration order is insertion order in python3.
This change removes setting the blamelist_pins attribute since its not
used anywhere and it will allow json.output to use plain dict.
Change-Id: Ie19b87d61903774454df2d395acf695eaa39b201
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4617204
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Auto-Submit: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Switch from Xcode 13.0 to 13.2.1 to update the macOS SDK from 11.3 to
12.1. This is needed for bots that use this receipe, like PDFium Mac
bots, after crrev.com/1154867 made macOS 12 mandatory. This also has the
effect of updating the iOS SDK from 15.0 to 15.2.
Change-Id: I4c91b2c23bd8eaacac6cef6afe71fed3533fd838
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4605813
Commit-Queue: Lei Zhang <thestig@chromium.org>
Reviewed-by: Lindsay Pasricha <lindsayw@chromium.org>
The logging & path expansion shouldn't trigger exceptions, so move
them out of the try block to reduce the scope for this.
Bug: b/285376658
Change-Id: I617d572d997f8eaafa222cdb6c1d984c459d9733
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4582289
Commit-Queue: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Makes the code a little simpler and avoids relying on the env directly.
Bug: b/285376658
Change-Id: Iae8300059b3418e0a9224eb555df6eb6cafa9437
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4582288
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
This change adds git cache's epoch timestamp to bot_update's output.properties. It achieves this by adding a marker file to the cache directory which stores the epoch time.
Design: go/query-bot-update-cache-age
Bug: 1448769
Change-Id: I610c6bca5ced7e2a0fe3ee8570decf0f135fe54c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4563726
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
(1) bot_update (http://shortn/_I814yt6r9U) will use got_revision_mapping
in these configs to create builder properties
{
'got_revision': <revision>
'got_revision_superroject': <revision>
}
(2)
recipe testing a change will use these configs to go from:
CL's project -> project's name in gclient solutions -> got_revision string
(e..g 'https...infra/infra_superproject' -> '.' -> 'got_revision_superproject'
(3) then use e.g. 'got_revision_superproject' to get the <revision> it should use from (1) and pass to wherever. in the presubmit recipes case, we're passing it as upstream to presubmit_support.py http://shortn/_BHBdN9p7ja
After this change we can add infra and internal presubmit tryjobs to the infra_superproject repo.
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: chromiumos
Bug: 1415507
Change-Id: Ia9804325c35b6071afa73d1b1099695ea6ae0b03
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4570829
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
This would add support for uploading/installing Xcode with MacOS13 adoption
Recipe-Nontrivial-Roll: build_limited
Bug: 1441931
Change-Id: I7496a80a6876fc4b0df712ce1634611f59240e67
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4566408
Reviewed-by: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Jeff Yoon <jeffyoon@google.com>
Commit-Queue: Yue She <yueshe@google.com>
Fixes this error:
http://shortn/_MMjWQ6UDNN
presubmit.execute() (http://shortn/_IqUEskOvZy) calls get_project_revision_properties (http://shortn/_2YWZfKEBWb).
the `project_name` passed in was '.' (the solution root) so get_project_revision_properties was returning nothing.
setting 'infra_internal' in repo_path_map leads to this code (http://shortn/_7cO431MLDm) setting `self._relative_root` to `infra_internal` as expected. So `project_name` above will be `infra_internal` not '.'
In this case, also seems to fix the empty commit patch issue, as the `git commit` gets executed in infra_internal now.
led run: http://shortn/_UM6iapZUkx
Bug: 1415507
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Change-Id: Ib457ded91ffe5b9041b8b1bb0345229bfd07d1f3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4545587
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
This changes all references to the LUCI_OMIT_PYTHON2 environment
variable to instead assume that it is `'true'`.
R=sokcevic, tikuta
Bug: 1441784
Recipe-Nontrivial-Roll: build
Change-Id: Iea161a372adb68cdcb330d131df2c19ca2fe7b37
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4522480
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
This is to deactivate `luci.buildbucket.omit_python2` experiment.
I'll submit this after https://crrev.com/c/4515085 is submitted.
Bug: 1401307, 1441784
Recipe-Nontrivial-Roll: build
Change-Id: I0aec58e9b96c961da21f0de0850c4078bb679c33
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4513661
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
By default during tests, the _get_footers_step call will return None for
the returned footers because it does not set any test data. This does
not match the actual behavior of the recipe because git_footers.py
returns an empty json object if the commit message has no footers. This
change updates it to return an empty dict instead of None and checks for
the cached footers value to be None instead of any non-true value so
that a commit message without footers doesn't get parsed multiple times.
Change-Id: I716a27e964eb92de138228df5cc0876322d82823
Recipe-Manual-Change: build
Recipe-Nontrivial-Roll: build_limited
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4492269
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
https://crrev.com/c/4420526 put a 10s timeout on all calls made via
this script.
But it appears there are a variety of endpoints that take more than
10s, the latest being "SubmitChange", eg:
https://ci.chromium.org/ui/p/chromeos/builders/release/release-R113-15393.B-orchestrator/25/overview
So instead of applying the timeout to everything, and then opting
certain long-running calls out of it, this just opts-in the
"GetChanges" call to the timeout. This should cover the failure mode
that was originally solved for chrome's bots. And also bump the
timeout to 30s since we don't trust all get-changes queries to
reliably finish in under 10s.
This also bumps the step level timeout for the query in tryserver
recipe_mod to 8 min since there could be ~5min worth of sleeping +
150s worth of waiting+timing-out. So worst case the step will now
take 8min to fully exhaust all timeouts/sleeps.
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Bug: b/278083716
Change-Id: Ib366e004e0bb07297ba732590d488cae779e38ba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4426524
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
When gerrit's availability drops, chrome's builds see an excessive
amount of failures, eg:
https://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359780/overviewhttps://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359594/overviewhttps://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359564/overviewhttps://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359630/overview
Seemingly all failures occur in either the `gerrit fetch current CL
info` step or the `gerrit changes` step. Both steps have a 60s timeout
and shell out to depot_tools' gerrit_client.py. That script essentially
makes a single http call to gerrit. That request has no configured
timeout. So when gerrit's MIA and the call hangs indefinitely, so too
will the step hang. 60s after that, the step timeout is reached, and the
entire build crashes with an infra-failure.
However, one single retry has been shown to sufficiently work around
at least one instance of that failure:
dea9a6eca26f690ce10a959701a8fb51ad/+/build.proto
So this incorporates timeouts into the requests made by
gerrit_util.py. Each request is given a 10s timeout, which should be
enough for most/all gerrit calls. (Both steps have a p90 of less than
1sec.) When a timeout is reached, the script will automatically retry
up to 4 times.
This also bumps the timeouts of the step calls to gerrit_client.py to
6min since the script can now take up to 5 minutes to fail, ie:
the sequence of sleeps is roughly 10s, 20s, 40s, 80s, 160s, which is
about 5min. So a 6min timeout should cover all those retries.
This also passes in "--verbose" to all step calls to this script, so
the logging that prints out the retry + sleep log lines will be
visible in build logs.
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Bug: 1432638
Change-Id: I9dc47f4beeda3783ae4f9152bd29ee441ac3e197
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4420526
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
This kwarg will display the command that would have been run, without
actually running it. I plan to use this kwarg on some ChromeOS staging
builders.
BUG=b:270142110
TEST=./recipes.py test train
Change-Id: I20194152e305dc6bae71a58b94de71d414839643
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4382316
Reviewed-by: Gavin Mak <gavinmak@google.com>
Auto-Submit: Greg Edelston <gredelston@google.com>
Commit-Queue: Greg Edelston <gredelston@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Gavin Mak
Josip Sokcevic
recipe-roller
Robert Iannucci
Yue She
Yiwei Zhang
Xinan Lin
Garrett Beaty
Lei Zhang
Arthur Eubanks
Takuto Ikuta
Mike Frysinger
Aravind Vasudevan
Joanna Wang
Ariel Zhang
Ben Pastene
Greg Edelston
Joanna Wang