Commit Graph

157 Commits (60c4b8810bdc3107a9a0c7046b79ee3a95df47fb)

Author SHA1 Message Date
Gavin Mak 60c4b8810b Reserve time from soft_deadline to upload traces if bot_update times out
The "upload traces" step is skipped if bot_update times out and is in
the grace_period. Reserve 1 minute to upload traces.

Bug: 1351547, 1456840
Change-Id: Idb191735a07d245575ff4c79375a976750522fa5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4651461
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
2 years ago
Gavin Mak 94761e3305 Fix bot_update _trace_dir
The traces dir isn't created yet and fails to upload traces.

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Bug: 1351547, 1456880
Change-Id: I635ad02168c918028a3bd4ad6f914f5835dad9b6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4633186
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2 years ago
Garrett Beaty 3ffad8166e Don't set blamelist_pins on the json output.
The blamelist_pins attribute is being set on the json output, which is a
dictionary. This only works right now because json.output currently uses
OrderedDict as the dict type. With removal of support for python2, dict
could be used since its iteration order is insertion order in python3.
This change removes setting the blamelist_pins attribute since its not
used anywhere and it will allow json.output to use plain dict.

Change-Id: Ie19b87d61903774454df2d395acf695eaa39b201
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4617204
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Auto-Submit: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
2 years ago
Gavin Mak 6fde53139b Upload git traces from bot_update
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Bug: 1351547
Change-Id: I1e018a8407c10ad4f9b6674d4b17ccae1e863389
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4568628
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
2 years ago
Joanna Wang 1affd9c35d Add arg in bot_update recipes api to override deps revisions.
Bug: 1415507
Change-Id: I0bb224eb2b9b4f37d56543c40612736c3c8322a1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4561776
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Ravi Mistry f7e510b2f7 Add download-topics support to bot_update
Tested by:
* `python recipes.py test run`
* Tested end-to-end by patching in this unsubmitted change in Skia's recipes with https://skia-review.googlesource.com/c/skia/+/532768

Bug: chromium:1319415
Change-Id: Ia1c9c495ef6482b3fdb494e1c1c9320541bcd0c5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3602901
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Ravi Mistry <rmistry@chromium.org>
3 years ago
Garrett Beaty 3b97fa826e Reland "Set a default got_revision property in the bot_update json output."
This is a reland of commit 053817260a

The change was not related to the errors that it was reverted for, so no
changes are necessary.

Original change's description:
> Set a default got_revision property in the bot_update json output.
>
> The bot_update.py script sets a got_revision property by default even if
> it is not present in the reverse revision mapping. This results in an
> uncaught exception when set_output_commit is set to True if got_revision
> isn't present in the reverse revision mapping, but it isn't caught until
> production because the test API doesn't match that behavior. This change
> updates the test API method to match the behavior of the script.
>
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited
> Recipe-Nontrivial-Roll: chromiumos
> Recipe-Nontrivial-Roll: infra
> Change-Id: Ideefa9d77d2a816ae66a2bb52737264ed3f5bcee
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3575361
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Garrett Beaty <gbeaty@google.com>
> Auto-Submit: Garrett Beaty <gbeaty@google.com>

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Change-Id: I80efeee8a2b951df43b00d89bb596dc2cf2fcec7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3582182
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Garrett Beaty <gbeaty@google.com>
3 years ago
Josip Sokcevic a0382d39be Revert "Set a default got_revision property in the bot_update json output."
This reverts commit 053817260a.

Reason for revert: we reverted got_revision change, and this change
itself may be causing some other failues.

Original change's description:
> Set a default got_revision property in the bot_update json output.
>
> The bot_update.py script sets a got_revision property by default even if
> it is not present in the reverse revision mapping. This results in an
> uncaught exception when set_output_commit is set to True if got_revision
> isn't present in the reverse revision mapping, but it isn't caught until
> production because the test API doesn't match that behavior. This change
> updates the test API method to match the behavior of the script.
>
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited
> Recipe-Nontrivial-Roll: chromiumos
> Recipe-Nontrivial-Roll: infra
> Change-Id: Ideefa9d77d2a816ae66a2bb52737264ed3f5bcee
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3575361
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Garrett Beaty <gbeaty@google.com>
> Auto-Submit: Garrett Beaty <gbeaty@google.com>

Change-Id: Ie975dfe39e7b8997336761e12f53f3046359d825
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3578820
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
3 years ago
Garrett Beaty 053817260a Set a default got_revision property in the bot_update json output.
The bot_update.py script sets a got_revision property by default even if
it is not present in the reverse revision mapping. This results in an
uncaught exception when set_output_commit is set to True if got_revision
isn't present in the reverse revision mapping, but it isn't caught until
production because the test API doesn't match that behavior. This change
updates the test API method to match the behavior of the script.

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Change-Id: Ideefa9d77d2a816ae66a2bb52737264ed3f5bcee
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3575361
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Auto-Submit: Garrett Beaty <gbeaty@google.com>
3 years ago
Jack Neus 5c2d472e0c Revert "bot_update: change set_output_commit to True by default"
This reverts commit ffeef9001e.

Reason for revert: Breaks ChromeOS builders, e.g. go/bbid/8817528282713702353

Original change's description:
> bot_update: change set_output_commit to True by default
>
> This is for https://crbug.com/1310066#c7
>
> Bug: 1310066

> Change-Id: I874e1a4c2ca53a7da04d6c24533429d7508088a8
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3565259
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
> Auto-Submit: Takuto Ikuta <tikuta@chromium.org>

Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: build
Bug: 1310066
Change-Id: Id6b32f71ba8824ad3e748d0b52603ea1548096e5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3575359
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Garrett Beaty <gbeaty@google.com>
Owners-Override: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
3 years ago
Takuto Ikuta ffeef9001e bot_update: change set_output_commit to True by default
This is for https://crbug.com/1310066#c7

Bug: 1310066
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: build
Change-Id: I874e1a4c2ca53a7da04d6c24533429d7508088a8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3565259
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
3 years ago
Stephanie Kim 3481645827 Check if json exists in ensure_checkout's finally block
The step_result json can currently be empty if the orchestrator build
is canceled while running bot_update. Trying to do
step_result.json.output in the finally block raises an "Uncaught
exception AttributeError".
Ex: https://ci.chromium.org/ui/p/chromium/builders/try/win10_chromium_x64_rel_ng/1121568/overview

This change checks for the json attr before trying to read json.output.

Bug: 1305332
Change-Id: I1dc277642ece1b96144be6381a7c4b8426cb2bc6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3518372
Reviewed-by: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
3 years ago
Ravi Mistry 6b28c1ddef Replace python3 with vpython in bot_update/api.py
For context see comments 5 and 6 here: https://bugs.chromium.org/p/chromium/issues/detail?id=1298922#c6

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Bug: chromium:1298922
Change-Id: I23eaec3c104c7b52653732b6fd9d58d175d78069
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3489327
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Ravi Mistry <rmistry@chromium.org>
3 years ago
Josip Sokcevic 289dedd36b Remove unused arguments from bot_update recipe
gclient removed syntax validation flag in 2020:
https://crrev.com/c/2083589, so this can safely be removed.

R=aravindvasudev@google.com, gavinmak@google.com

Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: build
Change-Id: I2aced6cdb01d571ca361cc57e7c8715c9ca40552
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3419278
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
3 years ago
Josip Sokcevic e1b48631b2 Remove deprecated arguments in bot_update recipe
R=gavinmak@google.com

Change-Id: I37bbdf02bfd32577224224a3114171810a7318fd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3416677
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
3 years ago
Josip Sokcevic ed6aa2b0e8 Reland "Use py3 in bot_update and gclient recipe_modules"
This is a reland of bb88492ff3

Original change's description:
> Use py3 in bot_update and gclient recipe_modules
>
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited
> Recipe-Nontrivial-Roll: chrome_release
> Recipe-Nontrivial-Roll: chromiumos
> Recipe-Nontrivial-Roll: infra
> Bug: 1289280
> Change-Id: Id61bacf4acef319f7ea22161081b632677ca363c
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3402082
> Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Bug: 1289280, 1289504
Change-Id: I2f2f7244568b1fdfb960d5cdfa3937175ac1ed2f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3414742
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
3 years ago
Brian Ryner 6bfbe35cce Revert "Use py3 in bot_update and gclient recipe_modules"
This reverts commit bb88492ff3.

Reason for revert: Broke bot_update: https://logs.chromium.org/logs/infra/buildbucket/cr-buildbucket/8824463357494497185/+/u/bot_update/stdout

Original change's description:
> Use py3 in bot_update and gclient recipe_modules
>
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited
> Recipe-Nontrivial-Roll: chrome_release
> Recipe-Nontrivial-Roll: chromiumos
> Recipe-Nontrivial-Roll: infra
> Bug: 1289280
> Change-Id: Id61bacf4acef319f7ea22161081b632677ca363c
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3402082
> Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Bug: 1289280
Change-Id: I371d599b17b42edd7b1aae581695e75a80ebae83
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3403476
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
3 years ago
Josip Sokcevic bb88492ff3 Use py3 in bot_update and gclient recipe_modules
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Bug: 1289280
Change-Id: Id61bacf4acef319f7ea22161081b632677ca363c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3402082
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
3 years ago
Garrett Beaty 92c05b07e4 Produce blamelist pins in a consistent order.
The blamelist pins are produced by iterating over a dictionary which
yields a different order between python2 and python3. The blamelist pins
will now be produced with a consistent ordering between python versions
by sorting on the revision name.

Change-Id: Ieb6c4dfe0aacea83c4a86a3a10380cc2f3839114
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3352250
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Garrett Beaty <gbeaty@google.com>
3 years ago
Josip Sokcevic 756e98f5aa Mark patch failures as regular failures, not infra
Patch failures shouldn't be treated as infra_failures (as long as patch
was successfully downloaded).

Prior to https://crbug.com/1207685, bot_update patch failurs were set as
successful step to work around buildbucket plugin requirements.

One alternative approach is to add necessary information into patch
failure step, by editing its presentation.

R=gavinmak@google.com

Bug: 1207685, 1255228
Change-Id: I696356d40a80536c3efbca99091536b8b25deb40
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3219890
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Yuanjun Huang 8f09549ffc [python3] Solve soft errors in the recipes of git module
These soft errors will become hard failures and block the proto module
to be marked PY2+3 in upstream recipe engine repo.


Bug: 1227140
Change-Id: I0e5fdebe7e7cf43552b525d63c12f82c96c49902
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3152784
Commit-Queue: Yuanjun Huang <yuanjunh@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
4 years ago
Josip Sokcevic bdc7adc6f7 Remove bot_update assumption that HEAD is main
There is no need to override refs/heads/main to be HEAD. This may not
always be true, there is no benefits of doing so and it doesn't play
well with git_cache which will set HEAD to global default branch.

R=gavinmak@google.com

Bug: 1238084
Recipe-Manual-Change: build_limited
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Change-Id: I21ab3463baaf7f86562ab0c65c06c3ac0962a08c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3088295
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
4 years ago
Josip Sokcevic ce332a47a9 Use main as default branch in all recipes
Recipe-Manual-Change: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Change-Id: I89008bc07b1c3de13d9c6dd64e723fe6dc8d01d6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3015262
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Anthony Polito <apolito@google.com>
4 years ago
Edward Lesmes 6d099d543d bot_update: Don't collect metrics if not running as part of a build.
When running using led, DEPOT_TOOLS_REPORT_BUILD might not be set, and
bots will try to collect metrics.

Set DEPOT_TOOLS_COLLECT_METRICS so metrics are not collected on this
case.

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: chromiumos
Change-Id: I8b1083b31ef6daf87d24544a990bee863543aa1d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2946935
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Gavin Mak 6b0a611c2c Make patch failures from bot_update fail in ensure_checkout
Patch failures in bot_update return exit code 87 and 88, which are
ignored by ensure_checkout. This causes the bot_update step in recipes
to be successful despite a patch failure. This change removes these exit
codes from the accepted return codes for this step.

Bug: 1207685
Change-Id: I7bd71732a99184c96196659c6953e4d4b610c9b9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2893686
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
4 years ago
Edward Lesmes 8c9a5b84cb bot_update: Add support for experiments.
Bug: 1199853
Change-Id: I0616f90a0cd7e4478f40ac7eaf5dbccca788d517
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2898001
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Edward Lesmes 68c71a34e1 bot_update: Set DEPOT_TOOLS_REPORT_BUILD to report bot metrics.
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: chrome_release
Change-Id: I704ac523a3212ea30061948f8a9971a373d5394f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2861214
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
4 years ago
Josip Sokcevic 3d9cab4fa8 Increase bot_update http low speed time
Windows builders have issues fetching from chromium/src, and increasing
timeout may help - recommended by git team.

Bug: 1198581
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: build
Change-Id: Ic650dea2a175e9642a6d3d5b8f1c54fa9979e12a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2823020
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Josip Sokcevic 76d0e6aca7 [bot_update] Resolve revision only if patching
If bot_update is not patching, it shouldn't look into gitiles patch
information.

R=ehmaldonado@google.com

Bug: 1147625
Recipe-Nontrivial-Roll: build
Change-Id: I111f0dc1c36d588341c02930bf1cda1f5b8cc2e7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2644199
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Josip Sokcevic 361ccce124 Revert "Support main branch in bot_update"
This reverts commit e7a31f42cd.

Reason for revert: has typo and doesn't work if typo is fixed

Original change's description:
> Support main branch in bot_update
>
> Change-Id: I58e88e9505779c309ac0af9d5b78b45feaa75543
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528669
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>

TBR=ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Change-Id: Ib6b266250b854e2554249b956766214be84ec522
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2643961
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Garrett Beaty e856b6bba8 Enable set_output_commit to be used with RevisionFallbackChain.
When set_output_commit is True and the repo associated with the
got_revision property does not use commit positions, buiders whose
gclient config uses RevisionFallbackChain for that repo will raise an
exception when attempting set the output commit because a string is
expected for the revision value, not a RevisionFallbackChain object.
This changes the set_output_commit code to resolve the revision so that
the output commit can be set in those cases.

Bug: 1159749, 1141886
Change-Id: Iddf1d49871028d9500eefb0eed07e36ce7e6b0d6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2612223
Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Xinan Lin 26ced49089 Improve the readability on variable name added in crrev/c/2528670
Bug: 1146487
Change-Id: I60476ac60ede4810f63403791386128508631db3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2551386
Commit-Queue: Xinan Lin <linxinan@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Xinan Lin <linxinan@chromium.org>
4 years ago
Xinan Lin 5589652282 Revert "Improve the setup logic for tryserver"
This reverts commit 4447336100.

Reason for revert: Breaks infra repo. See crrev.com/c/2548285

Original change's description:
> Improve the setup logic for tryserver
>
> If the given change does not have a valid gerrit host name, stop
> initializing the tryserver module.
> Also polished the variable name and docstr added in crrev/c/2528670.
>
> Recipe-Nontrivial-Roll: infra
> Bug:1146487
> Change-Id: I38193b25d37c724fd41d898eb020237fbf7cecdf
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2544000
> Commit-Queue: Xinan Lin <linxinan@chromium.org>
> Reviewed-by: Nodir Turakulov <nodir@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>

TBR=nodir@chromium.org,tandrii@google.com,recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com,linxinan@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

Change-Id: I3871866454adab816ab5549f8273fe16af2d9adb
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1146487
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2548409
Reviewed-by: Xinan Lin <linxinan@chromium.org>
Commit-Queue: Xinan Lin <linxinan@chromium.org>
5 years ago
Xinan Lin 4447336100 Improve the setup logic for tryserver
If the given change does not have a valid gerrit host name, stop
initializing the tryserver module.
Also polished the variable name and docstr added in crrev/c/2528670.

Recipe-Nontrivial-Roll: infra
Bug:1146487
Change-Id: I38193b25d37c724fd41d898eb020237fbf7cecdf
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2544000
Commit-Queue: Xinan Lin <linxinan@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
5 years ago
Xinan Lin da73999f79 Make bot_update and tryserver module not crash for multiple changes
Currently bot_update and tryserver crashes at initialization step if
the input contains multiple gerrit changes. This is not friendly for
CrOS CLs, e.g. crrev.com/c/2536761.

This CL moves the “crash” out of initialization, thus the consumer
of these modules could control which gerrit change to pass to bot_update
and tryserver.

This CL does NOT enable the modules here to support multiple patchset,
but allows consumers to choose one.

Bug:1146487
Change-Id: Iec46a8041189912e34da9bc5e8ff4611123ec9e0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528670
Commit-Queue: Xinan Lin <linxinan@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
5 years ago
Edward Lesmes 17cd53de67 Revert "Return HEAD when ref is refs/heads/main"
This reverts commit cbdf0c1e41.

Reason for revert: 
Broke tint bots

From log:
https://logs.chromium.org/logs/tint/buildbucket/cr-buildbucket.appspot.com/8864003131025797600/+/steps/bot_update/0/stdout 

Bot page:
https://ci.chromium.org/p/tint/builders/try/linux-clang-rel-x64/b8864003131025797600?

Original change's description:
> Return HEAD when ref is refs/heads/main
>
> R=​ehmaldonado@chromium.org
>
> Change-Id: I67b40cf48f0f9880e0f055a83d768bbefd704892
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528118
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
> Reviewed-by: Dirk Pranke <dpranke@google.com>

TBR=dpranke@google.com,ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

Bug: 1147494
Change-Id: Ieb21e15161ccc267f5d453a0196b1f0a2b43f366
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2529700
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Josip Sokcevic cbdf0c1e41 Return HEAD when ref is refs/heads/main
R=ehmaldonado@chromium.org

Change-Id: I67b40cf48f0f9880e0f055a83d768bbefd704892
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528118
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
5 years ago
Josip Sokcevic e7a31f42cd Support main branch in bot_update
Change-Id: I58e88e9505779c309ac0af9d5b78b45feaa75543
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528669
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Quinten Yearsley b674f8a277 Edit depot_tools recipe_modules docstrings for consistency
Change-Id: I454e9c90d3159598ae34d97d22c96b9d20d55f80
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2521656
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
5 years ago
Stephanie Kim da6894f7a9 Add enforce_fetch to ensure_checkout cmd arg
The enforce_fetch kwarg was being ignored. This change adds the
--enforce_fetch to the resources/bot_update cmd.
Confirmed here through led: b852fa5e4bb77d6fb2aa69ec87ee57b5d2/+/steps/bot_update/0/logs/execution_details/0

Bug: 1129235
Change-Id: I78c5d62841fda6dafbb709301e26c5047df2d58d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2444114
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim b677e98529 Reland "Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache""
This is a reland of 3774c589cb

Added missing options.enforce_fetch arg

Confirmed via a fake test roll with this CL's patch commit that the
bot_update step passes with this change:
https://chromium-review.googlesource.com/c/chromium/tools/build/+/2443649

Original change's description:
> Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache"
>
> This is a reland of f38f54f2f1
>
> Original change's description:
> > Add optional enforce_fetch kwarg to fetch and refresh git cache
> >
> > Allows builders to do a force refresh of the git cache to ensure a new
> > fetch happens. android-binary-size needs this to get an accurate diff
> > of the passed in src revision and the applied patch:
> > https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
> >
> > Bug: 1129235
> > Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> > Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
>
> Bug: 1129235
> Change-Id: I034a91a8b57821d08b10042f7fc94d32350540ba
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443235
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>

Bug: 1129235
Change-Id: Ibbd4535f8529243d2dc1d4d32881d8a61efe374a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443574
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim d628ffbd8f Revert "Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache""
This reverts commit 3774c589cb.

Reason for revert: ea31a9d6c2771701fcaa11b47a2d735c19/+/steps/bot_update/0/stdout Going to revert and test with led before landing another "fix"

Original change's description:
> Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache"
>
> This is a reland of f38f54f2f1
>
> Original change's description:
> > Add optional enforce_fetch kwarg to fetch and refresh git cache
> >
> > Allows builders to do a force refresh of the git cache to ensure a new
> > fetch happens. android-binary-size needs this to get an accurate diff
> > of the passed in src revision and the applied patch:
> > https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
> >
> > Bug: 1129235
> > Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> > Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
>
> Bug: 1129235
> Change-Id: I034a91a8b57821d08b10042f7fc94d32350540ba
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443235
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>

TBR=tandrii@google.com,ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com,kimstephanie@google.com

Change-Id: I305215eed4128b13c9b5b524f4cbf3203037d582
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1129235
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443573
Reviewed-by: Stephanie Kim <kimstephanie@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim 3774c589cb Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache"
This is a reland of f38f54f2f1

Original change's description:
> Add optional enforce_fetch kwarg to fetch and refresh git cache
>
> Allows builders to do a force refresh of the git cache to ensure a new
> fetch happens. android-binary-size needs this to get an accurate diff
> of the passed in src revision and the applied patch:
> https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
>
> Bug: 1129235
> Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>

Bug: 1129235
Change-Id: I034a91a8b57821d08b10042f7fc94d32350540ba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443235
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim 98218e9219 Revert "Add optional enforce_fetch kwarg to fetch and refresh git cache"
This reverts commit f38f54f2f1.

Reason for revert: Seeing errors about missing args 007f2e16445872d9237cec0f72bb239fd6/+/steps/bot_update/0/stdout

Original change's description:
> Add optional enforce_fetch kwarg to fetch and refresh git cache
>
> Allows builders to do a force refresh of the git cache to ensure a new
> fetch happens. android-binary-size needs this to get an accurate diff
> of the passed in src revision and the applied patch:
> https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
>
> Bug: 1129235
> Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>

TBR=tandrii@google.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com,kimstephanie@google.com

Change-Id: Ib867742c624f085d3dd56422c2d63c15fdc84f06
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1129235
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443231
Reviewed-by: Stephanie Kim <kimstephanie@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim f38f54f2f1 Add optional enforce_fetch kwarg to fetch and refresh git cache
Allows builders to do a force refresh of the git cache to ensure a new
fetch happens. android-binary-size needs this to get an accurate diff
of the passed in src revision and the applied patch:
https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19

Bug: 1129235
Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
5 years ago
Garrett Beaty 71198cd4a2 Handle failures in the bot_update script.
Currently, if the bot_update script fails, then recipe code that makes
assumptions about the returned output JSON cause exceptions to be
raised. This fixes a couple instances in the code so that the bot_update
failure should surface instead of an uncaught exception.

Change-Id: I031e613841b03e5eb4f6d9ca7adb5085088a3077
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2405708
Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Garrett Beaty 6c484875cd Ensure all keys in the bot_update revisions dict use forward slash.
On Windows, the path that gets computed from the gitiles commit uses
backslashes, which won't match the format of the paths that
bot_update.py uses.

Bug: 1047893
Change-Id: Ie1f6c070558fe00d6f4b1f2fd453e430b21a55ed
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2380590
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
5 years ago
Garrett Beaty 6081b23b29 Modify bot_update to support setting blamelist pins.
This adds a parameter to bot_update.ensure_checkout that will cause
blamelist pins to be set for all of the repos whose revision is not
determined by DEPS files: the repo for the gitiles input, the repo for
each solution in the gclient config and any repos with the revision set
by the revisions field of the gclient config.

Bug: 1047893
Change-Id: Ie3eee41dbf15be014185b6f255ee3d4cfd40b4e4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2293514
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Robert Iannucci d56d2c282f Remove source_manifest functionality from depot_tools.
This is essentially defunct; We will want to add something like this
back in, but it's so sparsely used right now that it's better to just
remove it and then add it back properly when we're ready to support
it across all of LUCI.

R=yiwzhang@google.com

Bug: 1093959
Change-Id: I72673515d860234daa3b01514ac38168ebb4825a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2250403
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
5 years ago
Josip Sokcevic a01816b0ba Set git low speed detection on bot_update
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: release_scripts
Recipe-Nontrivial-Roll: skia
Change-Id: Iacf276bd53fdca37994b6d6fe1e5db71f844a3fe
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2147833
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago