Commit Graph

280 Commits (11f53d6949438d0735c36998460871dbd1e84bb5)

Author SHA1 Message Date
Gavin Mak 11f53d6949 Reland "Add python3 support to bot_update recipe module"
This is a reland of edb6e8c51f

Original change's description:
> Add python3 support to bot_update recipe module
>
> Bug: 1227140
> Change-Id: Id4938c966da54957f84f59d0aef0c68c381ad5f6
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3016032
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Gavin Mak <gavinmak@google.com>

Bug: 1227140
Change-Id: I2d35d29f87236ce10ecabca41712cf1c90de8136
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3022310
Reviewed-by: Anthony Polito <apolito@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
4 years ago
Gavin Mak f6228e1583 Revert "Add python3 support to bot_update recipe module"
This reverts commit edb6e8c51f.

Reason for revert: missing some py3 changes 

Original change's description:
> Add python3 support to bot_update recipe module
>
> Bug: 1227140
> Change-Id: Id4938c966da54957f84f59d0aef0c68c381ad5f6
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3016032
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Gavin Mak <gavinmak@google.com>

Bug: 1227140
Change-Id: If0852908b112ff6bdd4152189c7cb198eebd2c22
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3022309
Auto-Submit: Gavin Mak <gavinmak@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
4 years ago
Gavin Mak edb6e8c51f Add python3 support to bot_update recipe module
Bug: 1227140
Change-Id: Id4938c966da54957f84f59d0aef0c68c381ad5f6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3016032
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
4 years ago
Edward Lesmes 6d099d543d bot_update: Don't collect metrics if not running as part of a build.
When running using led, DEPOT_TOOLS_REPORT_BUILD might not be set, and
bots will try to collect metrics.

Set DEPOT_TOOLS_COLLECT_METRICS so metrics are not collected on this
case.

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: chromiumos
Change-Id: I8b1083b31ef6daf87d24544a990bee863543aa1d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2946935
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Edward Lesmes b508ecd932 bot_update: Initialize synced_solutions before syncing.
If sync fails, synced_solutions is not initialized, resulting in a
use-before-init error.

Bug: 1217352
Change-Id: I2367d0f8030faa571b5c2743d5d97aa4c0329a7b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2945663
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Edward Lesmes 9705687c25 bot_update: Add experiment to avoid fetching if checked out revision is recent enough.
If checked-out revision is less than 6 hours old, skip fetching latest
revision.

Bug: 1199853
Change-Id: Ifd2a61055de286fe6fd94a5d43f80a1885a6931c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2911015
Reviewed-by: Erik Staab <estaab@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Edward Lesmes 50cd6e416a Reland "bot_update: Don't use gclient sync output and rely on gclient revinfo."
This is a reland of 856ccef0e9

Use a tempfile instead of stdout for gclient revinfo output

Original change's description:
> bot_update: Don't use gclient sync output and rely on gclient revinfo.
>
> In preparation for skipping gclient sync if there are no DEPS changes.
>
> Bug: 1199853
> Change-Id: Ib9b4ab803bc574a384c661765cee5e4c1de5baae
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2912259
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

Bug: 1199853
Change-Id: Ib50c4e943cbb51e4a2463beee9b7d1dee824ad29
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2910950
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Stephen Martinis e2d7d8cd9b Revert "bot_update: Don't use gclient sync output and rely on gclient revinfo."
This reverts commit 856ccef0e9.

Reason for revert: Probably broke chrome os waterfall

Original change's description:
> bot_update: Don't use gclient sync output and rely on gclient revinfo.
>
> In preparation for skipping gclient sync if there are no DEPS changes.
>
> Bug: 1199853
> Change-Id: Ib9b4ab803bc574a384c661765cee5e4c1de5baae
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2912259
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

Bug: 1199853
Change-Id: Ib893704dffc68b02dfa7f977e751e236003b7e0b
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2910949
Auto-Submit: Stephen Martinis <martiniss@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Avi Drissman <avi@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
Owners-Override: Avi Drissman <avi@chromium.org>
4 years ago
Edward Lesmes 856ccef0e9 bot_update: Don't use gclient sync output and rely on gclient revinfo.
In preparation for skipping gclient sync if there are no DEPS changes.

Bug: 1199853
Change-Id: Ib9b4ab803bc574a384c661765cee5e4c1de5baae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2912259
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Edward Lesmes 6097a92db0 bot_update: Remove more unused arguments.
Change-Id: I00b32eb109cdff7e138ef5d9497eda77c33188f3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2912295
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
4 years ago
Edward Lesmes da8a67b497 bot_update: Remove unused arguments/code.
Change-Id: I4a6811f4dd3c5330eed821acd2dc4855aabcf483
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2911202
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Gavin Mak 6b0a611c2c Make patch failures from bot_update fail in ensure_checkout
Patch failures in bot_update return exit code 87 and 88, which are
ignored by ensure_checkout. This causes the bot_update step in recipes
to be successful despite a patch failure. This change removes these exit
codes from the accepted return codes for this step.

Bug: 1207685
Change-Id: I7bd71732a99184c96196659c6953e4d4b610c9b9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2893686
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
4 years ago
Edward Lesmes 8c9a5b84cb bot_update: Add support for experiments.
Bug: 1199853
Change-Id: I0616f90a0cd7e4478f40ac7eaf5dbccca788d517
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2898001
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Anthony Polito 542224ebd6 Replace processObservers with a class that doesn't constantly recreate threads
I'm not saying this is why bot_update is failing, but I never trust python threads, especially not at scale. I also think the new implementation is easier to understand and less likely to have subtle mistakes if it is extended later

Change-Id: I29be55594a400fda845ea31893789104373fd0f6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2892185
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Anthony Polito <apolito@google.com>
4 years ago
recipe-roller b47c95a58a Roll recipe dependencies (nontrivial).
This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.

Please review the expectation changes, and LGTM+CQ.

recipe_engine:
f4c9a77634
  f4c9a77 (olivernewman@google.com)
      [buildbucket] Make default git_ref consistent

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.

R=iannucci@chromium.org, tandrii@google.com

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I6b5526f445bbb7337ce8d18e10e855f7a91c587f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2885939
Auto-Submit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
4 years ago
Anthony Polito f7157ceea8 add a big of debugging info to figure out more abour bot_update failures
Change-Id: If324a6af154c4ded3b4c8d5e07a4740722318a59
BUG:1206377
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2889824
Auto-Submit: Anthony Polito <apolito@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Edward Lesmes 68c71a34e1 bot_update: Set DEPOT_TOOLS_REPORT_BUILD to report bot metrics.
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: chrome_release
Change-Id: I704ac523a3212ea30061948f8a9971a373d5394f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2861214
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
4 years ago
Edward Lesmes 3600164d99 recipes: Specify refs/heads/master instead of origin/master for gclient config.
Bug: 1200657
Change-Id: I9362a66d12d75d26e1236f9d8516b03561fb9383
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2837689
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Edward Lesmes 07a683430c Reland "git-cache: Add option to fetch commits."
This is a reland of 4c67f856f0
Issues should have been fixed by crrev.com/c/2838026

Original change's description:
> git-cache: Add option to fetch commits.
>
> Add option to git cache to fetch commits.
> And use it in bot_update and gclient sync to make sure
> the needed commits are present on the checkout.
>
> Change-Id: I9e90da9e3be6e7bacf714b22bf0b735463e655b6
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2829942
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

Change-Id: Ie5a29737f5a75d28bc7c5c2f6cb99ec7f87cd9e8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2841046
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Edward Lesmes 93d80d8686 Revert "git-cache: Add option to fetch commits."
This reverts commit 4c67f856f0.

Reason for revert: crbug.com/1200657

Original change's description:
> git-cache: Add option to fetch commits.
>
> Add option to git cache to fetch commits.
> And use it in bot_update and gclient sync to make sure
> the needed commits are present on the checkout.
>
> Change-Id: I9e90da9e3be6e7bacf714b22bf0b735463e655b6
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2829942
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

Change-Id: I31a1defb26119bcb46894ab4084d626b65c45fed
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2837413
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
4 years ago
Edward Lesmes 4c67f856f0 git-cache: Add option to fetch commits.
Add option to git cache to fetch commits.
And use it in bot_update and gclient sync to make sure
the needed commits are present on the checkout.

Change-Id: I9e90da9e3be6e7bacf714b22bf0b735463e655b6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2829942
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Edward Lesmes 1dbd65b18a bot_update: Fetch required commit/refs directly.
Change-Id: Ie6bee1743859df364025350fc1a607f331346bd7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2825426
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Josip Sokcevic 784f5c890e Revert "Remove TAGS_REFSPEC if no_fetch_tags is true"
This reverts commit 9a94a95e65.

Reason for revert: crbug/1199445

Original change's description:
> Remove TAGS_REFSPEC if no_fetch_tags is true
>
> Bug: 1188627
> Change-Id: I6ee14ee9976b9aab60ce409ddd17f018ad966055
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2797933
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Gavin Mak <gavinmak@google.com>

Bug: 1188627, 1199445
Change-Id: I776639709221d8e0505315c451f847d6cf592b62
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2828772
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Josip Sokcevic 3d9cab4fa8 Increase bot_update http low speed time
Windows builders have issues fetching from chromium/src, and increasing
timeout may help - recommended by git team.

Bug: 1198581
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: build
Change-Id: Ic650dea2a175e9642a6d3d5b8f1c54fa9979e12a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2823020
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Haiyang Pan 057831ef1f Recipe: Add timeout for fetching gerrit commit message in tryserver API
In crbug.com/1196159, a build got stuck at "gerrit changes" step for 3
hours. It would be good to add a timeout so that it fails early.

Also adjust the timeout for get_changes to 60s, from 600s

Bug: 1196159
Change-Id: Id2327ab40dd2fd8d95aaad6bb724f7ed06e6c244
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: build
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2811899
Commit-Queue: Haiyang Pan <hypan@google.com>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
4 years ago
Gavin Mak 9a94a95e65 Remove TAGS_REFSPEC if no_fetch_tags is true
Bug: 1188627
Change-Id: I6ee14ee9976b9aab60ce409ddd17f018ad966055
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2797933
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
4 years ago
recipe-roller 68a0333ec7 Roll recipe dependencies (nontrivial).
This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.

Please review the expectation changes, and LGTM+CQ.

recipe_engine:
3faa1bab09
  3faa1ba (yiwzhang@google.com)
      cq: Use proto Output properties in cq recipe_module

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.

R=dpranke@google.com

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I5cb2fc5927de1cecad74f2c7a25be875983556a2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2781195
Auto-Submit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Dirk Pranke <dpranke@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
4 years ago
Gavin Mak 2b29d0f8e6 Fix invalid reference: refs/remotes/origin/HEAD error
Change-Id: Iac298a7a14b5c0b55fd199ec120fe5c37ef88fdc
Bug: 1178944
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2757129
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
4 years ago
Josip Sokcevic 2c9dbb89fb Revert "Add previous_got_revision and previous_got_revision_cp to output"
This reverts commit 6d0c0ffef5.

Reason for revert: this appears to  break builders if reference
is a commit hash:

git log --format=%B -n1 HEAD failed with code 128

Original change's description:
> Add previous_got_revision and previous_got_revision_cp to output
>
> As chrome browser infra works to improve CQ times, we would like to
> easily glance at builds and see how many new commits were checked out.
> Currently you have to open up bot_update stdout to see the previous
> HEAD revision and go from there. Having this conveniently outputted
> would be very convenient.
>
> If the bot has previously checked out the repo, output which revision
> and commit position were checked out.
>
> example LED build showing new output: https://ci.chromium.org/swarming/task/51592e46d6a70e10?server=chromium-swarm.appspot.com
>
> Change-Id: Ib8a6f152a593446ce4082477bc9ada44f1fd1fc3
> Bug: 1170568
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2644829
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>

TBR=infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com,kimstephanie@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: 1170568
Change-Id: I50d6ef6802c2eb1614cca804391c7826a79a2cf9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2657023
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Stephanie Kim 6d0c0ffef5 Add previous_got_revision and previous_got_revision_cp to output
As chrome browser infra works to improve CQ times, we would like to
easily glance at builds and see how many new commits were checked out.
Currently you have to open up bot_update stdout to see the previous
HEAD revision and go from there. Having this conveniently outputted
would be very convenient.

If the bot has previously checked out the repo, output which revision
and commit position were checked out.

example LED build showing new output: https://ci.chromium.org/swarming/task/51592e46d6a70e10?server=chromium-swarm.appspot.com

Change-Id: Ib8a6f152a593446ce4082477bc9ada44f1fd1fc3
Bug: 1170568
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2644829
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
4 years ago
Josip Sokcevic 76d0e6aca7 [bot_update] Resolve revision only if patching
If bot_update is not patching, it shouldn't look into gitiles patch
information.

R=ehmaldonado@google.com

Bug: 1147625
Recipe-Nontrivial-Roll: build
Change-Id: I111f0dc1c36d588341c02930bf1cda1f5b8cc2e7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2644199
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Edward Lesmes d686b1d1c8 bot_update: Set encrypted debug headers when retrying fetch.
Bug: b/178502800
Bug: 1170806
Change-Id: If66cf06b9954520ddd909dce869e2a15da3055ea
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2650965
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Josip Sokcevic 361ccce124 Revert "Support main branch in bot_update"
This reverts commit e7a31f42cd.

Reason for revert: has typo and doesn't work if typo is fixed

Original change's description:
> Support main branch in bot_update
>
> Change-Id: I58e88e9505779c309ac0af9d5b78b45feaa75543
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528669
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>

TBR=ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Change-Id: Ib6b266250b854e2554249b956766214be84ec522
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2643961
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Josip Sokcevic 60b433cf0b Enable packfile offloading on all builders
R=ehmaldonado@chromium.org

Bug: 1136986
Change-Id: I012f0d29589460b8cc7809c9c011811dff476a02
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2616898
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Garrett Beaty e856b6bba8 Enable set_output_commit to be used with RevisionFallbackChain.
When set_output_commit is True and the repo associated with the
got_revision property does not use commit positions, buiders whose
gclient config uses RevisionFallbackChain for that repo will raise an
exception when attempting set the output commit because a string is
expected for the revision value, not a RevisionFallbackChain object.
This changes the set_output_commit code to resolve the revision so that
the output commit can be set in those cases.

Bug: 1159749, 1141886
Change-Id: Iddf1d49871028d9500eefb0eed07e36ce7e6b0d6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2612223
Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Josip Sokcevic d8fe52139f Check PACKFILE_OFFLOADING env variable
If set, set git config fetch.uriprotocols to https. This will be
initially set by codesearch builders. Once tested, this env variable can
be removed.

R=ehmaldonaldo@chromium.org

Bug: 1136986
Change-Id: I1c3da87bdbc74d18585ed00ffdc2cf21b84b16fa
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2591808
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Xinan Lin 26ced49089 Improve the readability on variable name added in crrev/c/2528670
Bug: 1146487
Change-Id: I60476ac60ede4810f63403791386128508631db3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2551386
Commit-Queue: Xinan Lin <linxinan@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Xinan Lin <linxinan@chromium.org>
5 years ago
Xinan Lin 5589652282 Revert "Improve the setup logic for tryserver"
This reverts commit 4447336100.

Reason for revert: Breaks infra repo. See crrev.com/c/2548285

Original change's description:
> Improve the setup logic for tryserver
>
> If the given change does not have a valid gerrit host name, stop
> initializing the tryserver module.
> Also polished the variable name and docstr added in crrev/c/2528670.
>
> Recipe-Nontrivial-Roll: infra
> Bug:1146487
> Change-Id: I38193b25d37c724fd41d898eb020237fbf7cecdf
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2544000
> Commit-Queue: Xinan Lin <linxinan@chromium.org>
> Reviewed-by: Nodir Turakulov <nodir@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>

TBR=nodir@chromium.org,tandrii@google.com,recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com,linxinan@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

Change-Id: I3871866454adab816ab5549f8273fe16af2d9adb
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1146487
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2548409
Reviewed-by: Xinan Lin <linxinan@chromium.org>
Commit-Queue: Xinan Lin <linxinan@chromium.org>
5 years ago
Xinan Lin 4447336100 Improve the setup logic for tryserver
If the given change does not have a valid gerrit host name, stop
initializing the tryserver module.
Also polished the variable name and docstr added in crrev/c/2528670.

Recipe-Nontrivial-Roll: infra
Bug:1146487
Change-Id: I38193b25d37c724fd41d898eb020237fbf7cecdf
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2544000
Commit-Queue: Xinan Lin <linxinan@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
5 years ago
Xinan Lin da73999f79 Make bot_update and tryserver module not crash for multiple changes
Currently bot_update and tryserver crashes at initialization step if
the input contains multiple gerrit changes. This is not friendly for
CrOS CLs, e.g. crrev.com/c/2536761.

This CL moves the “crash” out of initialization, thus the consumer
of these modules could control which gerrit change to pass to bot_update
and tryserver.

This CL does NOT enable the modules here to support multiple patchset,
but allows consumers to choose one.

Bug:1146487
Change-Id: Iec46a8041189912e34da9bc5e8ff4611123ec9e0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528670
Commit-Queue: Xinan Lin <linxinan@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
5 years ago
Edward Lesmes 17cd53de67 Revert "Return HEAD when ref is refs/heads/main"
This reverts commit cbdf0c1e41.

Reason for revert: 
Broke tint bots

From log:
https://logs.chromium.org/logs/tint/buildbucket/cr-buildbucket.appspot.com/8864003131025797600/+/steps/bot_update/0/stdout 

Bot page:
https://ci.chromium.org/p/tint/builders/try/linux-clang-rel-x64/b8864003131025797600?

Original change's description:
> Return HEAD when ref is refs/heads/main
>
> R=​ehmaldonado@chromium.org
>
> Change-Id: I67b40cf48f0f9880e0f055a83d768bbefd704892
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528118
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
> Reviewed-by: Dirk Pranke <dpranke@google.com>

TBR=dpranke@google.com,ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

Bug: 1147494
Change-Id: Ieb21e15161ccc267f5d453a0196b1f0a2b43f366
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2529700
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Josip Sokcevic cbdf0c1e41 Return HEAD when ref is refs/heads/main
R=ehmaldonado@chromium.org

Change-Id: I67b40cf48f0f9880e0f055a83d768bbefd704892
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528118
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
5 years ago
Josip Sokcevic e7a31f42cd Support main branch in bot_update
Change-Id: I58e88e9505779c309ac0af9d5b78b45feaa75543
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528669
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Quinten Yearsley b674f8a277 Edit depot_tools recipe_modules docstrings for consistency
Change-Id: I454e9c90d3159598ae34d97d22c96b9d20d55f80
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2521656
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
5 years ago
Stephanie Kim da6894f7a9 Add enforce_fetch to ensure_checkout cmd arg
The enforce_fetch kwarg was being ignored. This change adds the
--enforce_fetch to the resources/bot_update cmd.
Confirmed here through led: b852fa5e4bb77d6fb2aa69ec87ee57b5d2/+/steps/bot_update/0/logs/execution_details/0

Bug: 1129235
Change-Id: I78c5d62841fda6dafbb709301e26c5047df2d58d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2444114
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim b677e98529 Reland "Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache""
This is a reland of 3774c589cb

Added missing options.enforce_fetch arg

Confirmed via a fake test roll with this CL's patch commit that the
bot_update step passes with this change:
https://chromium-review.googlesource.com/c/chromium/tools/build/+/2443649

Original change's description:
> Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache"
>
> This is a reland of f38f54f2f1
>
> Original change's description:
> > Add optional enforce_fetch kwarg to fetch and refresh git cache
> >
> > Allows builders to do a force refresh of the git cache to ensure a new
> > fetch happens. android-binary-size needs this to get an accurate diff
> > of the passed in src revision and the applied patch:
> > https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
> >
> > Bug: 1129235
> > Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> > Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
>
> Bug: 1129235
> Change-Id: I034a91a8b57821d08b10042f7fc94d32350540ba
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443235
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>

Bug: 1129235
Change-Id: Ibbd4535f8529243d2dc1d4d32881d8a61efe374a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443574
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim d628ffbd8f Revert "Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache""
This reverts commit 3774c589cb.

Reason for revert: ea31a9d6c2771701fcaa11b47a2d735c19/+/steps/bot_update/0/stdout Going to revert and test with led before landing another "fix"

Original change's description:
> Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache"
>
> This is a reland of f38f54f2f1
>
> Original change's description:
> > Add optional enforce_fetch kwarg to fetch and refresh git cache
> >
> > Allows builders to do a force refresh of the git cache to ensure a new
> > fetch happens. android-binary-size needs this to get an accurate diff
> > of the passed in src revision and the applied patch:
> > https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
> >
> > Bug: 1129235
> > Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> > Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
>
> Bug: 1129235
> Change-Id: I034a91a8b57821d08b10042f7fc94d32350540ba
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443235
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>

TBR=tandrii@google.com,ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com,kimstephanie@google.com

Change-Id: I305215eed4128b13c9b5b524f4cbf3203037d582
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1129235
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443573
Reviewed-by: Stephanie Kim <kimstephanie@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim 3774c589cb Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache"
This is a reland of f38f54f2f1

Original change's description:
> Add optional enforce_fetch kwarg to fetch and refresh git cache
>
> Allows builders to do a force refresh of the git cache to ensure a new
> fetch happens. android-binary-size needs this to get an accurate diff
> of the passed in src revision and the applied patch:
> https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
>
> Bug: 1129235
> Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>

Bug: 1129235
Change-Id: I034a91a8b57821d08b10042f7fc94d32350540ba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443235
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim 98218e9219 Revert "Add optional enforce_fetch kwarg to fetch and refresh git cache"
This reverts commit f38f54f2f1.

Reason for revert: Seeing errors about missing args 007f2e16445872d9237cec0f72bb239fd6/+/steps/bot_update/0/stdout

Original change's description:
> Add optional enforce_fetch kwarg to fetch and refresh git cache
>
> Allows builders to do a force refresh of the git cache to ensure a new
> fetch happens. android-binary-size needs this to get an accurate diff
> of the passed in src revision and the applied patch:
> https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
>
> Bug: 1129235
> Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>

TBR=tandrii@google.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com,kimstephanie@google.com

Change-Id: Ib867742c624f085d3dd56422c2d63c15fdc84f06
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1129235
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443231
Reviewed-by: Stephanie Kim <kimstephanie@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim f38f54f2f1 Add optional enforce_fetch kwarg to fetch and refresh git cache
Allows builders to do a force refresh of the git cache to ensure a new
fetch happens. android-binary-size needs this to get an accurate diff
of the passed in src revision and the applied patch:
https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19

Bug: 1129235
Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
5 years ago