The logging & path expansion shouldn't trigger exceptions, so move
them out of the try block to reduce the scope for this.
Bug: b/285376658
Change-Id: I617d572d997f8eaafa222cdb6c1d984c459d9733
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4582289
Commit-Queue: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Makes the code a little simpler and avoids relying on the env directly.
Bug: b/285376658
Change-Id: Iae8300059b3418e0a9224eb555df6eb6cafa9437
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4582288
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
This change adds git cache's epoch timestamp to bot_update's output.properties. It achieves this by adding a marker file to the cache directory which stores the epoch time.
Design: go/query-bot-update-cache-age
Bug: 1448769
Change-Id: I610c6bca5ced7e2a0fe3ee8570decf0f135fe54c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4563726
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
(1) bot_update (http://shortn/_I814yt6r9U) will use got_revision_mapping
in these configs to create builder properties
{
'got_revision': <revision>
'got_revision_superroject': <revision>
}
(2)
recipe testing a change will use these configs to go from:
CL's project -> project's name in gclient solutions -> got_revision string
(e..g 'https...infra/infra_superproject' -> '.' -> 'got_revision_superproject'
(3) then use e.g. 'got_revision_superproject' to get the <revision> it should use from (1) and pass to wherever. in the presubmit recipes case, we're passing it as upstream to presubmit_support.py http://shortn/_BHBdN9p7ja
After this change we can add infra and internal presubmit tryjobs to the infra_superproject repo.
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: chromiumos
Bug: 1415507
Change-Id: Ia9804325c35b6071afa73d1b1099695ea6ae0b03
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4570829
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
This would add support for uploading/installing Xcode with MacOS13 adoption
Recipe-Nontrivial-Roll: build_limited
Bug: 1441931
Change-Id: I7496a80a6876fc4b0df712ce1634611f59240e67
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4566408
Reviewed-by: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Jeff Yoon <jeffyoon@google.com>
Commit-Queue: Yue She <yueshe@google.com>
Fixes this error:
http://shortn/_MMjWQ6UDNN
presubmit.execute() (http://shortn/_IqUEskOvZy) calls get_project_revision_properties (http://shortn/_2YWZfKEBWb).
the `project_name` passed in was '.' (the solution root) so get_project_revision_properties was returning nothing.
setting 'infra_internal' in repo_path_map leads to this code (http://shortn/_7cO431MLDm) setting `self._relative_root` to `infra_internal` as expected. So `project_name` above will be `infra_internal` not '.'
In this case, also seems to fix the empty commit patch issue, as the `git commit` gets executed in infra_internal now.
led run: http://shortn/_UM6iapZUkx
Bug: 1415507
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Change-Id: Ib457ded91ffe5b9041b8b1bb0345229bfd07d1f3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4545587
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
This changes all references to the LUCI_OMIT_PYTHON2 environment
variable to instead assume that it is `'true'`.
R=sokcevic, tikuta
Bug: 1441784
Recipe-Nontrivial-Roll: build
Change-Id: Iea161a372adb68cdcb330d131df2c19ca2fe7b37
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4522480
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
This is to deactivate `luci.buildbucket.omit_python2` experiment.
I'll submit this after https://crrev.com/c/4515085 is submitted.
Bug: 1401307, 1441784
Recipe-Nontrivial-Roll: build
Change-Id: I0aec58e9b96c961da21f0de0850c4078bb679c33
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4513661
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
By default during tests, the _get_footers_step call will return None for
the returned footers because it does not set any test data. This does
not match the actual behavior of the recipe because git_footers.py
returns an empty json object if the commit message has no footers. This
change updates it to return an empty dict instead of None and checks for
the cached footers value to be None instead of any non-true value so
that a commit message without footers doesn't get parsed multiple times.
Change-Id: I716a27e964eb92de138228df5cc0876322d82823
Recipe-Manual-Change: build
Recipe-Nontrivial-Roll: build_limited
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4492269
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
https://crrev.com/c/4420526 put a 10s timeout on all calls made via
this script.
But it appears there are a variety of endpoints that take more than
10s, the latest being "SubmitChange", eg:
https://ci.chromium.org/ui/p/chromeos/builders/release/release-R113-15393.B-orchestrator/25/overview
So instead of applying the timeout to everything, and then opting
certain long-running calls out of it, this just opts-in the
"GetChanges" call to the timeout. This should cover the failure mode
that was originally solved for chrome's bots. And also bump the
timeout to 30s since we don't trust all get-changes queries to
reliably finish in under 10s.
This also bumps the step level timeout for the query in tryserver
recipe_mod to 8 min since there could be ~5min worth of sleeping +
150s worth of waiting+timing-out. So worst case the step will now
take 8min to fully exhaust all timeouts/sleeps.
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Bug: b/278083716
Change-Id: Ib366e004e0bb07297ba732590d488cae779e38ba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4426524
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
When gerrit's availability drops, chrome's builds see an excessive
amount of failures, eg:
https://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359780/overviewhttps://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359594/overviewhttps://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359564/overviewhttps://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359630/overview
Seemingly all failures occur in either the `gerrit fetch current CL
info` step or the `gerrit changes` step. Both steps have a 60s timeout
and shell out to depot_tools' gerrit_client.py. That script essentially
makes a single http call to gerrit. That request has no configured
timeout. So when gerrit's MIA and the call hangs indefinitely, so too
will the step hang. 60s after that, the step timeout is reached, and the
entire build crashes with an infra-failure.
However, one single retry has been shown to sufficiently work around
at least one instance of that failure:
dea9a6eca26f690ce10a959701a8fb51ad/+/build.proto
So this incorporates timeouts into the requests made by
gerrit_util.py. Each request is given a 10s timeout, which should be
enough for most/all gerrit calls. (Both steps have a p90 of less than
1sec.) When a timeout is reached, the script will automatically retry
up to 4 times.
This also bumps the timeouts of the step calls to gerrit_client.py to
6min since the script can now take up to 5 minutes to fail, ie:
the sequence of sleeps is roughly 10s, 20s, 40s, 80s, 160s, which is
about 5min. So a 6min timeout should cover all those retries.
This also passes in "--verbose" to all step calls to this script, so
the logging that prints out the retry + sleep log lines will be
visible in build logs.
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Bug: 1432638
Change-Id: I9dc47f4beeda3783ae4f9152bd29ee441ac3e197
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4420526
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
This kwarg will display the command that would have been run, without
actually running it. I plan to use this kwarg on some ChromeOS staging
builders.
BUG=b:270142110
TEST=./recipes.py test train
Change-Id: I20194152e305dc6bae71a58b94de71d414839643
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4382316
Reviewed-by: Gavin Mak <gavinmak@google.com>
Auto-Submit: Greg Edelston <gredelston@google.com>
Commit-Queue: Greg Edelston <gredelston@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Removes a TODO to remove the patch_text argument and its uses from
get_footers and related methods. The TODO indicated that it was
only used in example code, and thus should be handled another way.
However, it turns out that there is at least one legitimate use of
thus functionality downstream to retrieve footers from CL
descriptions earlier revisions.
Fixed: 1179039
Change-Id: I79996f6053a6769043943de39735b2b5d7ee02ee
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4283342
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
This reverts commit 19794fe91f.
Reason for revert: Doesn't fix the cache.
Original change's description:
> Reset origin/HEAD when it points to master
>
> This change tries resetting origin/HEAD when it points to master and the ref cannot be found. Post master->main migration, some cached repositories seem to point origin/HEAD to master.
>
> Bug: 1418866
> Change-Id: I83942901b105cace4d51deeac7e400f98d200168
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4296441
> Reviewed-by: Fumitoshi Ukai <ukai@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Bug: 1418866
Change-Id: I4c912f8401afb6fb0c2149c277778bdee9fa5d76
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4304240
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
This change tries resetting origin/HEAD when it points to master and the ref cannot be found. Post master->main migration, some cached repositories seem to point origin/HEAD to master.
Bug: 1418866
Change-Id: I83942901b105cace4d51deeac7e400f98d200168
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4296441
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Makes the git_cl recipe module use vpython3 instead of vpython, as
the latter breaks when running on builders with the omit_python2
experiment.
Bug: 1366965
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Change-Id: I2306e97dbc7b75201024b5dcf1892787185d5f77
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3968801
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
The python module is deprecated in favor of using the step module with
an explicit python binary. Switching the code to use the step module and
remove DEPS on recipe_engine/python removes warnings when running recipe
tests in the repo.
Change-Id: If3c9d5d6aff5aab69898b535399accbef92f5c0b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3880624
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Takuto Ikuta
Mike Frysinger
Aravind Vasudevan
Gavin Mak
Joanna Wang
Yue She
Robert Iannucci
Garrett Beaty
Ariel Zhang
Ben Pastene
Greg Edelston
Joanna Wang
Brian Sheedy
chrome-bot
Junji Watanabe
Josip Sokcevic
Michael Achenbach
Stephen Martinis
Riley Wong
Zhaoyang Li