Commit Graph

323 Commits (c9f904f2051c80c4610be50abc02bd064a948e0c)

Author SHA1 Message Date
Stephen Martinis e2d7d8cd9b Revert "bot_update: Don't use gclient sync output and rely on gclient revinfo."
This reverts commit 856ccef0e9.

Reason for revert: Probably broke chrome os waterfall

Original change's description:
> bot_update: Don't use gclient sync output and rely on gclient revinfo.
>
> In preparation for skipping gclient sync if there are no DEPS changes.
>
> Bug: 1199853
> Change-Id: Ib9b4ab803bc574a384c661765cee5e4c1de5baae
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2912259
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

Bug: 1199853
Change-Id: Ib893704dffc68b02dfa7f977e751e236003b7e0b
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2910949
Auto-Submit: Stephen Martinis <martiniss@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Avi Drissman <avi@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
Owners-Override: Avi Drissman <avi@chromium.org>
4 years ago
Edward Lesmes 856ccef0e9 bot_update: Don't use gclient sync output and rely on gclient revinfo.
In preparation for skipping gclient sync if there are no DEPS changes.

Bug: 1199853
Change-Id: Ib9b4ab803bc574a384c661765cee5e4c1de5baae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2912259
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Edward Lesmes 6097a92db0 bot_update: Remove more unused arguments.
Change-Id: I00b32eb109cdff7e138ef5d9497eda77c33188f3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2912295
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
4 years ago
Edward Lesmes da8a67b497 bot_update: Remove unused arguments/code.
Change-Id: I4a6811f4dd3c5330eed821acd2dc4855aabcf483
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2911202
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Gavin Mak 6b0a611c2c Make patch failures from bot_update fail in ensure_checkout
Patch failures in bot_update return exit code 87 and 88, which are
ignored by ensure_checkout. This causes the bot_update step in recipes
to be successful despite a patch failure. This change removes these exit
codes from the accepted return codes for this step.

Bug: 1207685
Change-Id: I7bd71732a99184c96196659c6953e4d4b610c9b9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2893686
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
4 years ago
Edward Lesmes 8c9a5b84cb bot_update: Add support for experiments.
Bug: 1199853
Change-Id: I0616f90a0cd7e4478f40ac7eaf5dbccca788d517
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2898001
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Anthony Polito 542224ebd6 Replace processObservers with a class that doesn't constantly recreate threads
I'm not saying this is why bot_update is failing, but I never trust python threads, especially not at scale. I also think the new implementation is easier to understand and less likely to have subtle mistakes if it is extended later

Change-Id: I29be55594a400fda845ea31893789104373fd0f6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2892185
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Anthony Polito <apolito@google.com>
4 years ago
recipe-roller b47c95a58a Roll recipe dependencies (nontrivial).
This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.

Please review the expectation changes, and LGTM+CQ.

recipe_engine:
f4c9a77634
  f4c9a77 (olivernewman@google.com)
      [buildbucket] Make default git_ref consistent

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.

R=iannucci@chromium.org, tandrii@google.com

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I6b5526f445bbb7337ce8d18e10e855f7a91c587f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2885939
Auto-Submit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
4 years ago
Anthony Polito f7157ceea8 add a big of debugging info to figure out more abour bot_update failures
Change-Id: If324a6af154c4ded3b4c8d5e07a4740722318a59
BUG:1206377
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2889824
Auto-Submit: Anthony Polito <apolito@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Edward Lesmes 68c71a34e1 bot_update: Set DEPOT_TOOLS_REPORT_BUILD to report bot metrics.
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: chrome_release
Change-Id: I704ac523a3212ea30061948f8a9971a373d5394f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2861214
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
4 years ago
Edward Lesmes 3600164d99 recipes: Specify refs/heads/master instead of origin/master for gclient config.
Bug: 1200657
Change-Id: I9362a66d12d75d26e1236f9d8516b03561fb9383
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2837689
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Edward Lesmes 07a683430c Reland "git-cache: Add option to fetch commits."
This is a reland of 4c67f856f0
Issues should have been fixed by crrev.com/c/2838026

Original change's description:
> git-cache: Add option to fetch commits.
>
> Add option to git cache to fetch commits.
> And use it in bot_update and gclient sync to make sure
> the needed commits are present on the checkout.
>
> Change-Id: I9e90da9e3be6e7bacf714b22bf0b735463e655b6
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2829942
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

Change-Id: Ie5a29737f5a75d28bc7c5c2f6cb99ec7f87cd9e8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2841046
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Edward Lesmes 93d80d8686 Revert "git-cache: Add option to fetch commits."
This reverts commit 4c67f856f0.

Reason for revert: crbug.com/1200657

Original change's description:
> git-cache: Add option to fetch commits.
>
> Add option to git cache to fetch commits.
> And use it in bot_update and gclient sync to make sure
> the needed commits are present on the checkout.
>
> Change-Id: I9e90da9e3be6e7bacf714b22bf0b735463e655b6
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2829942
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

Change-Id: I31a1defb26119bcb46894ab4084d626b65c45fed
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2837413
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
4 years ago
Edward Lesmes 4c67f856f0 git-cache: Add option to fetch commits.
Add option to git cache to fetch commits.
And use it in bot_update and gclient sync to make sure
the needed commits are present on the checkout.

Change-Id: I9e90da9e3be6e7bacf714b22bf0b735463e655b6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2829942
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Edward Lesmes 1dbd65b18a bot_update: Fetch required commit/refs directly.
Change-Id: Ie6bee1743859df364025350fc1a607f331346bd7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2825426
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Josip Sokcevic 784f5c890e Revert "Remove TAGS_REFSPEC if no_fetch_tags is true"
This reverts commit 9a94a95e65.

Reason for revert: crbug/1199445

Original change's description:
> Remove TAGS_REFSPEC if no_fetch_tags is true
>
> Bug: 1188627
> Change-Id: I6ee14ee9976b9aab60ce409ddd17f018ad966055
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2797933
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Gavin Mak <gavinmak@google.com>

Bug: 1188627, 1199445
Change-Id: I776639709221d8e0505315c451f847d6cf592b62
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2828772
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Josip Sokcevic 3d9cab4fa8 Increase bot_update http low speed time
Windows builders have issues fetching from chromium/src, and increasing
timeout may help - recommended by git team.

Bug: 1198581
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: build
Change-Id: Ic650dea2a175e9642a6d3d5b8f1c54fa9979e12a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2823020
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Haiyang Pan 057831ef1f Recipe: Add timeout for fetching gerrit commit message in tryserver API
In crbug.com/1196159, a build got stuck at "gerrit changes" step for 3
hours. It would be good to add a timeout so that it fails early.

Also adjust the timeout for get_changes to 60s, from 600s

Bug: 1196159
Change-Id: Id2327ab40dd2fd8d95aaad6bb724f7ed06e6c244
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: build
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2811899
Commit-Queue: Haiyang Pan <hypan@google.com>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
4 years ago
Gavin Mak 9a94a95e65 Remove TAGS_REFSPEC if no_fetch_tags is true
Bug: 1188627
Change-Id: I6ee14ee9976b9aab60ce409ddd17f018ad966055
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2797933
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
4 years ago
recipe-roller 68a0333ec7 Roll recipe dependencies (nontrivial).
This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.

Please review the expectation changes, and LGTM+CQ.

recipe_engine:
3faa1bab09
  3faa1ba (yiwzhang@google.com)
      cq: Use proto Output properties in cq recipe_module

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.

R=dpranke@google.com

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I5cb2fc5927de1cecad74f2c7a25be875983556a2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2781195
Auto-Submit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Dirk Pranke <dpranke@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
4 years ago
Gavin Mak 2b29d0f8e6 Fix invalid reference: refs/remotes/origin/HEAD error
Change-Id: Iac298a7a14b5c0b55fd199ec120fe5c37ef88fdc
Bug: 1178944
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2757129
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
4 years ago
Josip Sokcevic 2c9dbb89fb Revert "Add previous_got_revision and previous_got_revision_cp to output"
This reverts commit 6d0c0ffef5.

Reason for revert: this appears to  break builders if reference
is a commit hash:

git log --format=%B -n1 HEAD failed with code 128

Original change's description:
> Add previous_got_revision and previous_got_revision_cp to output
>
> As chrome browser infra works to improve CQ times, we would like to
> easily glance at builds and see how many new commits were checked out.
> Currently you have to open up bot_update stdout to see the previous
> HEAD revision and go from there. Having this conveniently outputted
> would be very convenient.
>
> If the bot has previously checked out the repo, output which revision
> and commit position were checked out.
>
> example LED build showing new output: https://ci.chromium.org/swarming/task/51592e46d6a70e10?server=chromium-swarm.appspot.com
>
> Change-Id: Ib8a6f152a593446ce4082477bc9ada44f1fd1fc3
> Bug: 1170568
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2644829
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>

TBR=infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com,kimstephanie@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: 1170568
Change-Id: I50d6ef6802c2eb1614cca804391c7826a79a2cf9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2657023
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Stephanie Kim 6d0c0ffef5 Add previous_got_revision and previous_got_revision_cp to output
As chrome browser infra works to improve CQ times, we would like to
easily glance at builds and see how many new commits were checked out.
Currently you have to open up bot_update stdout to see the previous
HEAD revision and go from there. Having this conveniently outputted
would be very convenient.

If the bot has previously checked out the repo, output which revision
and commit position were checked out.

example LED build showing new output: https://ci.chromium.org/swarming/task/51592e46d6a70e10?server=chromium-swarm.appspot.com

Change-Id: Ib8a6f152a593446ce4082477bc9ada44f1fd1fc3
Bug: 1170568
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2644829
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
4 years ago
Josip Sokcevic 76d0e6aca7 [bot_update] Resolve revision only if patching
If bot_update is not patching, it shouldn't look into gitiles patch
information.

R=ehmaldonado@google.com

Bug: 1147625
Recipe-Nontrivial-Roll: build
Change-Id: I111f0dc1c36d588341c02930bf1cda1f5b8cc2e7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2644199
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Edward Lesmes d686b1d1c8 bot_update: Set encrypted debug headers when retrying fetch.
Bug: b/178502800
Bug: 1170806
Change-Id: If66cf06b9954520ddd909dce869e2a15da3055ea
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2650965
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Josip Sokcevic 361ccce124 Revert "Support main branch in bot_update"
This reverts commit e7a31f42cd.

Reason for revert: has typo and doesn't work if typo is fixed

Original change's description:
> Support main branch in bot_update
>
> Change-Id: I58e88e9505779c309ac0af9d5b78b45feaa75543
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528669
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>

TBR=ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Change-Id: Ib6b266250b854e2554249b956766214be84ec522
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2643961
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Josip Sokcevic 60b433cf0b Enable packfile offloading on all builders
R=ehmaldonado@chromium.org

Bug: 1136986
Change-Id: I012f0d29589460b8cc7809c9c011811dff476a02
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2616898
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Garrett Beaty e856b6bba8 Enable set_output_commit to be used with RevisionFallbackChain.
When set_output_commit is True and the repo associated with the
got_revision property does not use commit positions, buiders whose
gclient config uses RevisionFallbackChain for that repo will raise an
exception when attempting set the output commit because a string is
expected for the revision value, not a RevisionFallbackChain object.
This changes the set_output_commit code to resolve the revision so that
the output commit can be set in those cases.

Bug: 1159749, 1141886
Change-Id: Iddf1d49871028d9500eefb0eed07e36ce7e6b0d6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2612223
Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Josip Sokcevic d8fe52139f Check PACKFILE_OFFLOADING env variable
If set, set git config fetch.uriprotocols to https. This will be
initially set by codesearch builders. Once tested, this env variable can
be removed.

R=ehmaldonaldo@chromium.org

Bug: 1136986
Change-Id: I1c3da87bdbc74d18585ed00ffdc2cf21b84b16fa
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2591808
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Xinan Lin 26ced49089 Improve the readability on variable name added in crrev/c/2528670
Bug: 1146487
Change-Id: I60476ac60ede4810f63403791386128508631db3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2551386
Commit-Queue: Xinan Lin <linxinan@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Xinan Lin <linxinan@chromium.org>
5 years ago
Xinan Lin 5589652282 Revert "Improve the setup logic for tryserver"
This reverts commit 4447336100.

Reason for revert: Breaks infra repo. See crrev.com/c/2548285

Original change's description:
> Improve the setup logic for tryserver
>
> If the given change does not have a valid gerrit host name, stop
> initializing the tryserver module.
> Also polished the variable name and docstr added in crrev/c/2528670.
>
> Recipe-Nontrivial-Roll: infra
> Bug:1146487
> Change-Id: I38193b25d37c724fd41d898eb020237fbf7cecdf
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2544000
> Commit-Queue: Xinan Lin <linxinan@chromium.org>
> Reviewed-by: Nodir Turakulov <nodir@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>

TBR=nodir@chromium.org,tandrii@google.com,recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com,linxinan@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

Change-Id: I3871866454adab816ab5549f8273fe16af2d9adb
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1146487
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2548409
Reviewed-by: Xinan Lin <linxinan@chromium.org>
Commit-Queue: Xinan Lin <linxinan@chromium.org>
5 years ago
Xinan Lin 4447336100 Improve the setup logic for tryserver
If the given change does not have a valid gerrit host name, stop
initializing the tryserver module.
Also polished the variable name and docstr added in crrev/c/2528670.

Recipe-Nontrivial-Roll: infra
Bug:1146487
Change-Id: I38193b25d37c724fd41d898eb020237fbf7cecdf
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2544000
Commit-Queue: Xinan Lin <linxinan@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
5 years ago
Xinan Lin da73999f79 Make bot_update and tryserver module not crash for multiple changes
Currently bot_update and tryserver crashes at initialization step if
the input contains multiple gerrit changes. This is not friendly for
CrOS CLs, e.g. crrev.com/c/2536761.

This CL moves the “crash” out of initialization, thus the consumer
of these modules could control which gerrit change to pass to bot_update
and tryserver.

This CL does NOT enable the modules here to support multiple patchset,
but allows consumers to choose one.

Bug:1146487
Change-Id: Iec46a8041189912e34da9bc5e8ff4611123ec9e0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528670
Commit-Queue: Xinan Lin <linxinan@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
5 years ago
Edward Lesmes 17cd53de67 Revert "Return HEAD when ref is refs/heads/main"
This reverts commit cbdf0c1e41.

Reason for revert: 
Broke tint bots

From log:
https://logs.chromium.org/logs/tint/buildbucket/cr-buildbucket.appspot.com/8864003131025797600/+/steps/bot_update/0/stdout 

Bot page:
https://ci.chromium.org/p/tint/builders/try/linux-clang-rel-x64/b8864003131025797600?

Original change's description:
> Return HEAD when ref is refs/heads/main
>
> R=​ehmaldonado@chromium.org
>
> Change-Id: I67b40cf48f0f9880e0f055a83d768bbefd704892
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528118
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
> Reviewed-by: Dirk Pranke <dpranke@google.com>

TBR=dpranke@google.com,ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

Bug: 1147494
Change-Id: Ieb21e15161ccc267f5d453a0196b1f0a2b43f366
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2529700
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Josip Sokcevic cbdf0c1e41 Return HEAD when ref is refs/heads/main
R=ehmaldonado@chromium.org

Change-Id: I67b40cf48f0f9880e0f055a83d768bbefd704892
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528118
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
5 years ago
Josip Sokcevic e7a31f42cd Support main branch in bot_update
Change-Id: I58e88e9505779c309ac0af9d5b78b45feaa75543
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528669
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Quinten Yearsley b674f8a277 Edit depot_tools recipe_modules docstrings for consistency
Change-Id: I454e9c90d3159598ae34d97d22c96b9d20d55f80
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2521656
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
5 years ago
Stephanie Kim da6894f7a9 Add enforce_fetch to ensure_checkout cmd arg
The enforce_fetch kwarg was being ignored. This change adds the
--enforce_fetch to the resources/bot_update cmd.
Confirmed here through led: b852fa5e4bb77d6fb2aa69ec87ee57b5d2/+/steps/bot_update/0/logs/execution_details/0

Bug: 1129235
Change-Id: I78c5d62841fda6dafbb709301e26c5047df2d58d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2444114
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim b677e98529 Reland "Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache""
This is a reland of 3774c589cb

Added missing options.enforce_fetch arg

Confirmed via a fake test roll with this CL's patch commit that the
bot_update step passes with this change:
https://chromium-review.googlesource.com/c/chromium/tools/build/+/2443649

Original change's description:
> Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache"
>
> This is a reland of f38f54f2f1
>
> Original change's description:
> > Add optional enforce_fetch kwarg to fetch and refresh git cache
> >
> > Allows builders to do a force refresh of the git cache to ensure a new
> > fetch happens. android-binary-size needs this to get an accurate diff
> > of the passed in src revision and the applied patch:
> > https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
> >
> > Bug: 1129235
> > Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> > Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
>
> Bug: 1129235
> Change-Id: I034a91a8b57821d08b10042f7fc94d32350540ba
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443235
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>

Bug: 1129235
Change-Id: Ibbd4535f8529243d2dc1d4d32881d8a61efe374a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443574
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim d628ffbd8f Revert "Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache""
This reverts commit 3774c589cb.

Reason for revert: ea31a9d6c2771701fcaa11b47a2d735c19/+/steps/bot_update/0/stdout Going to revert and test with led before landing another "fix"

Original change's description:
> Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache"
>
> This is a reland of f38f54f2f1
>
> Original change's description:
> > Add optional enforce_fetch kwarg to fetch and refresh git cache
> >
> > Allows builders to do a force refresh of the git cache to ensure a new
> > fetch happens. android-binary-size needs this to get an accurate diff
> > of the passed in src revision and the applied patch:
> > https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
> >
> > Bug: 1129235
> > Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> > Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
>
> Bug: 1129235
> Change-Id: I034a91a8b57821d08b10042f7fc94d32350540ba
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443235
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>

TBR=tandrii@google.com,ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com,kimstephanie@google.com

Change-Id: I305215eed4128b13c9b5b524f4cbf3203037d582
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1129235
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443573
Reviewed-by: Stephanie Kim <kimstephanie@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim 3774c589cb Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache"
This is a reland of f38f54f2f1

Original change's description:
> Add optional enforce_fetch kwarg to fetch and refresh git cache
>
> Allows builders to do a force refresh of the git cache to ensure a new
> fetch happens. android-binary-size needs this to get an accurate diff
> of the passed in src revision and the applied patch:
> https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
>
> Bug: 1129235
> Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>

Bug: 1129235
Change-Id: I034a91a8b57821d08b10042f7fc94d32350540ba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443235
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim 98218e9219 Revert "Add optional enforce_fetch kwarg to fetch and refresh git cache"
This reverts commit f38f54f2f1.

Reason for revert: Seeing errors about missing args 007f2e16445872d9237cec0f72bb239fd6/+/steps/bot_update/0/stdout

Original change's description:
> Add optional enforce_fetch kwarg to fetch and refresh git cache
>
> Allows builders to do a force refresh of the git cache to ensure a new
> fetch happens. android-binary-size needs this to get an accurate diff
> of the passed in src revision and the applied patch:
> https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
>
> Bug: 1129235
> Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>

TBR=tandrii@google.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com,kimstephanie@google.com

Change-Id: Ib867742c624f085d3dd56422c2d63c15fdc84f06
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1129235
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443231
Reviewed-by: Stephanie Kim <kimstephanie@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim f38f54f2f1 Add optional enforce_fetch kwarg to fetch and refresh git cache
Allows builders to do a force refresh of the git cache to ensure a new
fetch happens. android-binary-size needs this to get an accurate diff
of the passed in src revision and the applied patch:
https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19

Bug: 1129235
Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
5 years ago
Takuto Ikuta 5c4c28f111 [recipes] remove is_luci specification
This is always True now.

Bug: 1128847
Change-Id: Ief951e9312ea8bd13c98b4bbc87c18d7307bb43a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2413667
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
5 years ago
Andrii Shyshkalov 7b7bdcd075 tandrii: switch OWNERS to @google.com account.
R=tikuta
TBR=tandrii@chromium.org

Change-Id: I8ce4eb78e8ec2cf4c6d816b8a5ca63dbb0de66c6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2411985
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
5 years ago
Garrett Beaty 71198cd4a2 Handle failures in the bot_update script.
Currently, if the bot_update script fails, then recipe code that makes
assumptions about the returned output JSON cause exceptions to be
raised. This fixes a couple instances in the code so that the bot_update
failure should surface instead of an uncaught exception.

Change-Id: I031e613841b03e5eb4f6d9ca7adb5085088a3077
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2405708
Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Garrett Beaty 6c484875cd Ensure all keys in the bot_update revisions dict use forward slash.
On Windows, the path that gets computed from the gitiles commit uses
backslashes, which won't match the format of the paths that
bot_update.py uses.

Bug: 1047893
Change-Id: Ie1f6c070558fe00d6f4b1f2fd453e430b21a55ed
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2380590
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
5 years ago
Garrett Beaty 3e5d86a237 Generate revisions in json output for refs/ and origin/.
Right now the default test data for bot_update doesn't transform
revisions specifying refs for the revision except for refs starting with
refs/ on the first solution. This means that the output will just have
the ref name as the revision value, which isn't compatible with code
that requires an actual revision value.

Bug: 1047893
Change-Id: I016ad5e8db5a293dd5e6dacb5b8beba51e000949
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: chrome_release
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2325761
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes e4e1fb30c8 bot_update: Map refs to remote refs and don't checkout after clone.
Map refs to remote refs when fetching refs from origin
(e.g. refs/heads/master -> refs/remotes/origin/master)
so that a detached HEAD is not necessary.

Bug: 1104182
Change-Id: Icb751505de1bf443e5d45c4b37425223dbf6e088
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2310849
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Garrett Beaty 6081b23b29 Modify bot_update to support setting blamelist pins.
This adds a parameter to bot_update.ensure_checkout that will cause
blamelist pins to be set for all of the repos whose revision is not
determined by DEPS files: the repo for the gitiles input, the repo for
each solution in the gclient config and any repos with the revision set
by the revisions field of the gclient config.

Bug: 1047893
Change-Id: Ie3eee41dbf15be014185b6f255ee3d4cfd40b4e4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2293514
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Garrett Beaty c8c88f9c55 Use fixed revisions in bot_update's test API output_json.
Currently the output_json method generates revisions for projects that
are in the revision_mapping and only those projects appear in the
manifest and source_manifest, using the generated revision. This method
is used to construct the default json output for bot_update in tests.
This has 2 problems:
1. If a project has a fixed revision, its manifest entries and the
property used the generated revision, meaning the gitiles input revision
will not be used in the json output.
2. If a project has a fixed revision and isn't present in
revision_mapping then manifest entries aren't generated for it.

This changes the method to use the revisions in the fixed_revisions dict
and to create manifest entries for the projects that appear in the
fixed_revisions dict which should make fewer situations where
bot_update's output_json needs to be explicitly set.

Change-Id: I99de52edc9248c28e9d3aa3a04cc77bf9fccbe37
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: chrome_release
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2304644
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
5 years ago
Edward Lesmes 720e010ba1 Revert "bot_update: Refactor branch to detach and checkout."
This reverts commit b99e61c58a.

Reason for revert:
Doesn't work for refs/branch-heads/*

Original change's description:
> bot_update: Refactor branch to detach and checkout.
> 
> Refactor _git_checkout to use information from
> get_target_branch_and_revision to know what branch to checkout after
> cloning.
> 
> Bug: 1104182
> Change-Id: Ib3ba57ca0b6803f172b85121c2a4b123f17bfb8c
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2303291
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

TBR=ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

Change-Id: If51423df99fd9c164f8e42e0220ee2d9bc2a39f6
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1104182
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2310293
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes b99e61c58a bot_update: Refactor branch to detach and checkout.
Refactor _git_checkout to use information from
get_target_branch_and_revision to know what branch to checkout after
cloning.

Bug: 1104182
Change-Id: Ib3ba57ca0b6803f172b85121c2a4b123f17bfb8c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2303291
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes ed9e5acd61 bot_update: Don't checkout 'master' after clone.
Checkout 'HEAD' instead. This is needed to have a detached HEAD, so we
can fetch refs/heads/main:refs/heads/main, where main is the default
branch.

Bug: 1104182
Change-Id: I7b7457b21b7d98d29583555b62e27e96e900adb4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2300533
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
5 years ago
Greg Guterman 35dec5f127 Add gerrit_change_owner field to tryserver recipe api
Bug: 923016
Change-Id: I1d55fb2403306763a06b44d2ea65caa0014936d3
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: build
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2292235
Commit-Queue: Gregory Guterman <guterman@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Robert Iannucci d56d2c282f Remove source_manifest functionality from depot_tools.
This is essentially defunct; We will want to add something like this
back in, but it's so sparsely used right now that it's better to just
remove it and then add it back properly when we're ready to support
it across all of LUCI.

R=yiwzhang@google.com

Bug: 1093959
Change-Id: I72673515d860234daa3b01514ac38168ebb4825a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2250403
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
5 years ago
Arthur Wang 3f0096a72a Set DEPOT_TOOLS_UPDATE='0' assuming depot_tool do not auto update on bot
Bug: 1090603
Change-Id: Ib42c778c69588e44ce95b58eda5bc06b12550e9e
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: skia
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2228810
Commit-Queue: Arthur Wang <wuwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
5 years ago
Robert Iannucci 4504fd580a Remove infra_paths module.
We no longer need it to bridge path differences between buildbot and
LUCI, and it shouldn't be in use downstream now either.

R=machenbach@chromium.org, martiniss@chromium.org, tandrii@chromium.org, yiwzhang@google.com

Bug: 1015181
Change-Id: Iaba2100653ae9f06799d1d510bef993f7f710dc4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2220171
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
5 years ago
Josip Sokcevic 8280ae5e0a Delete dirty bot_update workspace
File marker '.dirty_bot_checkout' indicates that bot_update checkout has
started. Such marker is removed on successful checkout.

If such marker is present before creation, it means previous run wasn't
successful and may have left workspace in a bad state. In that case,
delete working directory and continue with a clean checkout.

R=apolito@google.com, ehmaldonado@chromium.org

Bug: 1074260
Change-Id: Ic089eedc26f7c269d5342fae366899571d8f8d41
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2186273
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
5 years ago
Josip Sokcevic d24ae26520 Remove curl verbose printing
R=apolito@google.com, ehmaldonado@chromium.org

Change-Id: Ie453bb95208561a09a5f1f62cb9543e31cbe031f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2171923
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Josip Sokcevic 350a913fad Terminate stale bot_update process
Introduce a process observer that terminates child process if there is
not stdout activity. It can be overridden by an environment variable.

R=apolito@google.com, ehmaldonado@chromium.org

Bug: 1074355
Change-Id: I11de9d29e716587614cf336725c8d4a368a9d61d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2167220
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Josip Sokcevic d70aa16fd5 Reland "Print ps in intervals while the process is running"
This is a reland of 50c6d8d348

Original change's description:
> Print ps in intervals while the process is running
>
> R=apolito@google.com, ehmaldonado@chromium.org
>
> Bug: 1074355
> Change-Id: Id2d5c8ca6f27fefb0fe8d769ae2e50dbaaddadbb
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2167221
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>

R=apolito@google.com, ehmaldonado@chromium.org

Bug: 1074355
Change-Id: I488b521e4a3b859c37f0bb1e21cad7d037f8f97e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2169392
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Josip Sokcevic 7d66c798bb Revert "Print ps in intervals while the process is running"
This reverts commit 50c6d8d348.

Reason for revert: self argument missing

Original change's description:
> Print ps in intervals while the process is running
> 
> R=​apolito@google.com, ehmaldonado@chromium.org
> 
> Bug: 1074355
> Change-Id: Id2d5c8ca6f27fefb0fe8d769ae2e50dbaaddadbb
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2167221
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>

TBR=ehmaldonado@chromium.org,apolito@google.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

Change-Id: I3c90576d0c6403f0d2f9119876a400c1380a8e14
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1074355
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2169391
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Josip Sokcevic 50c6d8d348 Print ps in intervals while the process is running
R=apolito@google.com, ehmaldonado@chromium.org

Bug: 1074355
Change-Id: Id2d5c8ca6f27fefb0fe8d769ae2e50dbaaddadbb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2167221
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Josip Sokcevic a01816b0ba Set git low speed detection on bot_update
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: release_scripts
Recipe-Nontrivial-Roll: skia
Change-Id: Iacf276bd53fdca37994b6d6fe1e5db71f844a3fe
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2147833
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Quinten Yearsley 925cedba7b Run spellchecker on depot_tools
This was made by running `codespell` and `scspell`
and then checking the results.

Change-Id: I169fd5b40294f83015075b4a899fbca263821f25
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2144602
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Auto-Submit: Quinten Yearsley <qyearsley@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Andrii Shyshkalov ef7a33909e bot_update: tell CQ to not retry patch application failures.
R=erikchen

Recipe-Nontrivial-Roll: build
Change-Id: I171daa3d61b2c20f963b8d587df8662740d80e71
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2050537
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Erik Chen <erikchen@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Andrii Shyshkalov <tandrii@google.com>
5 years ago
John Budorick e9aeaee522 bot_update: only add input commit refs for gclient solution roots.
Bug: 1031805
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: infra
Change-Id: I6edd221900eadd4d621d6190e08b61fdbc2f8621
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1958706
Commit-Queue: John Budorick <jbudorick@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
6 years ago
Garrett Beaty 90c88f0973 Reland "Fetch the input commit ref when both ref and id are specified."
This is a reland of 72a245e4c6

The "git checkout master --detach" operation has been modified to run in the solution
directory.

Original change's description:
> Fetch the input commit ref when both ref and id are specified.
>
> This causes --refs refs/head/master to be added to bot_update calls when
> the input build message contains a commit id and the refs/head/master
> ref. This is an effective no-op.
>
> When bot_update clones a git repository, it results in HEAD referring to
> master in contrast to the non-clone case which results in a detached
> HEAD. This prevents fetching refs/heads/master, so detach the HEAD after
> cloning.
>
> Bug: 922150
> Change-Id: I357a78305020218e9f5d350832a706b8f2b27967
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Recipe-Nontrivial-Roll: infra
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1900735
> Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>

Bug: 922150
Change-Id: I7dc6ebed2dcbd880ff4d515f7e9fc6428ff24099
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: infra
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1910260
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
6 years ago
Stephen Martinis 1917f7a099 Revert "Fetch the input commit ref when both ref and id are specified."
This reverts commit 72a245e4c6.

Reason for revert: crbug.com/1022617

Original change's description:
> Fetch the input commit ref when both ref and id are specified.
> 
> This causes --refs refs/head/master to be added to bot_update calls when
> the input build message contains a commit id and the refs/head/master
> ref. This is an effective no-op.
> 
> When bot_update clones a git repository, it results in HEAD referring to
> master in contrast to the non-clone case which results in a detached
> HEAD. This prevents fetching refs/heads/master, so detach the HEAD after
> cloning.
> 
> Bug: 922150
> Change-Id: I357a78305020218e9f5d350832a706b8f2b27967
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Recipe-Nontrivial-Roll: infra
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1900735
> Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>

TBR=ehmaldonado@chromium.org,gbeaty@chromium.org

Change-Id: Ibd6c6ed272b0c98f82be50cefa521aaee0444f11
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 922150, 1022617
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1904099
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
6 years ago
Garrett Beaty 72a245e4c6 Fetch the input commit ref when both ref and id are specified.
This causes --refs refs/head/master to be added to bot_update calls when
the input build message contains a commit id and the refs/head/master
ref. This is an effective no-op.

When bot_update clones a git repository, it results in HEAD referring to
master in contrast to the non-clone case which results in a detached
HEAD. This prevents fetching refs/heads/master, so detach the HEAD after
cloning.

Bug: 922150
Change-Id: I357a78305020218e9f5d350832a706b8f2b27967
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: infra
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1900735
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
danakj 23247b9932 Don't fetch tags when fetching to deapply a patch.
Bug: 1019824
Change-Id: Ibec0b47067345150dbd2083265c49fb9b6769a3a
Recipe-Nontrivial-Roll: build
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1894408
Commit-Queue: danakj <danakj@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
6 years ago
danakj c41f72cf44 Allow the bot_update api to not fetch tags from the git server.
There are 3 layers modified, starting from the bottom up:

1. git_cache.py populate

Now takes a --no-fetch-tags option. If specified, the cache will not
fetch updated tags from the server by passing --no-tags to git fetch.
This prevents the server from sending all the tag refs. In chromium
this prevents significant time bottlenecks dealing with 10k+ tags.

2. bot_update.py

bot_update has to deal with multiple git repos, it has the root repo
that is checked out through git-cache, as well as repos synched via
DEPS and gclient.

The script now takes a --no_fetch_tags option. If specified,
the git-cache populate call will include --no-fetch-tags. Otherwise, it
won't. So this controls (for chromium) if fetches to the src.git server
are passed --no-tags.

3. bot_update/api.py

This is the entry point for recipes and also has to deal with multiple
git repos. The behaviour at this point is not changed from the default.
A |no_fetch_tags| parameter is added to ensure_checkout() that defaults
to False.


This CL is a refactor with no intended behavior change.

The next step will be to change the chromium build repo to pass True
for ensure_checkout(no_fetch_tags) on chromium trybots.

This represents solution #2 in
https://docs.google.com/document/d/1hDIunJjjfpmr50y3YnZ4o3aq1NZF4gJa1TS0p7AIL64/edit#

Bug: 1019824
Change-Id: I935430603299daa9e301a95a5184c0ce486fd912
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1894352
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Erik Chen <erikchen@chromium.org>
Commit-Queue: danakj <danakj@chromium.org>
Auto-Submit: danakj <danakj@chromium.org>
6 years ago
Haiyang Pan 7a77984331 Revert "Fetch the input commit ref when both ref and id are specified."
This reverts commit 512f92b731.

Reason for revert: bot_update is having infra failure on ci bots.
For example https://ci.chromium.org/p/chromium/builders/ci/Closure%20Compilation%20Linux/249684.

Original change's description:
> Fetch the input commit ref when both ref and id are specified.
>
> This causes --refs refs/head/master to be added to bot_update
> calls when the input build message contains a commit id and the
> refs/head/master ref. This is an effective no-op.
>
> Bug: 922150
> Change-Id: I119d9b3cede727713a0b66bff83b5ee4e1848790
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1884457
> Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
> Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>

TBR=tandrii@google.com,gbeaty@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: 922150
Change-Id: Ie9275c21827688f59449f052c768452a8652e139
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1891000
Commit-Queue: Haiyang Pan <hypan@google.com>
Reviewed-by: Haiyang Pan <hypan@google.com>
6 years ago
John Budorick 48823c2854 Revert "Add more git tracing."
This reverts commit 1e4dbf3f64.

Reason for revert: unlikely but possible cause of http://shortn/_sgL4PqICVB

Original change's description:
> Add more git tracing.
> 
> This CL adds more perf tracing to `git fetch` from bot_update.py and
> git_cache.py
> 
> Change-Id: I9b69c60b6c81fc0c5f23f82fcc889b4d45a27556
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: infra
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1888433
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
> Commit-Queue: Erik Chen <erikchen@chromium.org>
> Auto-Submit: Erik Chen <erikchen@chromium.org>

TBR=danakj@chromium.org,tandrii@google.com,erikchen@chromium.org

Change-Id: I03b33bf1532212f83a6b89b1de32fafb6b8aafc4
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1891001
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
6 years ago
Erik Chen 1e4dbf3f64 Add more git tracing.
This CL adds more perf tracing to `git fetch` from bot_update.py and
git_cache.py

Change-Id: I9b69c60b6c81fc0c5f23f82fcc889b4d45a27556
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1888433
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Erik Chen <erikchen@chromium.org>
Auto-Submit: Erik Chen <erikchen@chromium.org>
6 years ago
Garrett Beaty 512f92b731 Fetch the input commit ref when both ref and id are specified.
This causes --refs refs/head/master to be added to bot_update
calls when the input build message contains a commit id and the
refs/head/master ref. This is an effective no-op.

Bug: 922150
Change-Id: I119d9b3cede727713a0b66bff83b5ee4e1848790
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1884457
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
6 years ago
Edward Lemur b594247e39 Reland "depot_tools: Run recipe scripts using vpython."
Bug: 1002153
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: release_scripts
Change-Id: I21b3223c5fc8a24fdafcc0795db51322cfe65148
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1865252
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Michael Achenbach 61d3d4b0bd Revert "depot_tools: Run recipe scripts using vpython."
This reverts commit c6512213a7.

Reason for revert: The recipe-roll https://crrev.com/c/1859692 is
stuck on owners approval and blocks a P0 revert. Please watch and
assist the roll on next reland.

Original change's description:
> depot_tools: Run recipe scripts using vpython.
> 
> This is a reland of 88d7869db0
> 
> Original change's description:
> > Reland "depot_tools: Add httplib2 to .vpython"
> >
> > This is a reland of e1410883a3
> >
> > Use vpython to execute git_cl.py, gerrit_util.py and presubmit_support.py on recipes.
> >
> > Original change's description:
> > > depot_tools: Add httplib2 to .vpython
> > >
> > > Check that things won't break before further changes are made.
> > >
> > > Bug: 1002153
> > > Change-Id: I41866f26334bf9ec2732bc0f25007234a95130e4
> > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1854749
> > > Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
> > > Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
> > > Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
> >
> > Bug: 1002153
> > Recipe-Nontrivial-Roll: build
> > Recipe-Nontrivial-Roll: chromiumos
> > Recipe-Nontrivial-Roll: infra
> > Recipe-Nontrivial-Roll: skia
> > Recipe-Nontrivial-Roll: build_limited_scripts_slave
> > Recipe-Nontrivial-Roll: release_scripts
> > Change-Id: Id94057eae8830dec197257df3ea35c0c4ff946b7
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1856650
> > Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> 
> Bug: 1002153
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: skia
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Recipe-Nontrivial-Roll: release_scripts
> Change-Id: If071bfd2e4514a06deec2c0d4196b4953841b64f
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1859895
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Anthony Polito <apolito@google.com>
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>

TBR=tandrii@google.com,ehmaldonado@chromium.org,apolito@google.com

Change-Id: I140264534db7eec0e1baf19354155d714fb58ed3
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1002153
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1861974
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
6 years ago
Edward Lemur c6512213a7 depot_tools: Run recipe scripts using vpython.
This is a reland of 88d7869db0

Original change's description:
> Reland "depot_tools: Add httplib2 to .vpython"
>
> This is a reland of e1410883a3
>
> Use vpython to execute git_cl.py, gerrit_util.py and presubmit_support.py on recipes.
>
> Original change's description:
> > depot_tools: Add httplib2 to .vpython
> >
> > Check that things won't break before further changes are made.
> >
> > Bug: 1002153
> > Change-Id: I41866f26334bf9ec2732bc0f25007234a95130e4
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1854749
> > Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
> > Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
> > Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
>
> Bug: 1002153
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: chromiumos
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: skia
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Recipe-Nontrivial-Roll: release_scripts
> Change-Id: Id94057eae8830dec197257df3ea35c0c4ff946b7
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1856650
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

Bug: 1002153
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: release_scripts
Change-Id: If071bfd2e4514a06deec2c0d4196b4953841b64f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1859895
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
6 years ago
Erik Chen e16ffffc28 Add logging to bot_update.py and gclient_scm.py
Bug: chromium:1014216
Change-Id: I620f197fe26d8817ddb51e7a30ae774439d79bdf
Recipe-Nontrivial-Roll: chromiumos
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1860854
Commit-Queue: Erik Chen <erikchen@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Auto-Submit: Erik Chen <erikchen@chromium.org>
6 years ago
Nodir Turakulov 5abb9b7c5d Revert "Reland "depot_tools: Add httplib2 to .vpython""
This reverts commit 88d7869db0.

Reason for revert: Broke luci-go-presubmit. luci-go development has stopped. https://ci.chromium.org/p/infra/builders/try/luci-go-try-presubmit

Original change's description:
> Reland "depot_tools: Add httplib2 to .vpython"
> 
> This is a reland of e1410883a3
> 
> Use vpython to execute git_cl.py, gerrit_util.py and presubmit_support.py on recipes.
> 
> Original change's description:
> > depot_tools: Add httplib2 to .vpython
> >
> > Check that things won't break before further changes are made.
> >
> > Bug: 1002153
> > Change-Id: I41866f26334bf9ec2732bc0f25007234a95130e4
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1854749
> > Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
> > Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
> > Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
> 
> Bug: 1002153
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: chromiumos
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: skia
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Recipe-Nontrivial-Roll: release_scripts
> Change-Id: Id94057eae8830dec197257df3ea35c0c4ff946b7
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1856650
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

TBR=tandrii@google.com,ehmaldonado@chromium.org,apolito@google.com

Change-Id: Ieecf0bf9164a14542a70ee6343763781a098a4a8
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1002153
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1858280
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
6 years ago
Edward Lemur 88d7869db0 Reland "depot_tools: Add httplib2 to .vpython"
This is a reland of e1410883a3

Use vpython to execute git_cl.py, gerrit_util.py and presubmit_support.py on recipes.

Original change's description:
> depot_tools: Add httplib2 to .vpython
>
> Check that things won't break before further changes are made.
>
> Bug: 1002153
> Change-Id: I41866f26334bf9ec2732bc0f25007234a95130e4
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1854749
> Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>

Bug: 1002153
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: release_scripts
Change-Id: Id94057eae8830dec197257df3ea35c0c4ff946b7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1856650
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Marc-Antoine Ruel 8e57b4bc55 python3 improvements
Ran:
  vi $(git grep --name-only iteritems | grep -v third_party)
  vi $(git grep --name-only itervalues | grep -v third_party)
  vi $(git grep --name-only 'print ' | grep -v third_party)

and edited the files quickly with adhoc macros. Then ran in recipes/:
  ./recipes.py test train

There was only a small subset of files that had been updated to use
six.iteritems() and six.itervalues(). Since the dataset size that is
being used in gclient is small (pretty much always below 200 items),
it's better to just switch to .items() right away and take the temporary
performance hit, so that we don't need to come back to rewrite the code.

Recipe-Nontrivial-Roll: build
Bug: 984182
Change-Id: I5faf11486b66b0d73c9098ab0f2ce1b15a45c53e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1854900
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Marc-Antoine Ruel <maruel@chromium.org>
6 years ago
Andrii Shyshkalov 2025b98401 recipes: update to stop using legacy properties.
The only exception is 'revision' of gclient's weird RevisionResolver,
which will be tackled in https://crbug.com/949334

R=ehmaldonado

Bug: 877161
Change-Id: Ic65a2d13578802cf093f2a1cfb4002fe97509b3b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1818973
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
6 years ago
Nico Weber 7735f52c6d Reland "Set core.trustctime=false during git checkout operations."
This is a reland of 3f79763629.
Last time it broke a skia test. The test has since been fixed
in https://crbug.com/1005388

Original change's description:
> Set core.trustctime=false during git checkout operations.
>
> This should be faster and more general than running
> `git index --refresh` over all git repositories we can find.
>
> See "Proposal: Setting core.trustctime=false on Chromium's bots"
> on infra-dev (linked to from bug 330461 comment 45).
>
> Bug: 330461,642711,746763,740301.
> Change-Id: I7c35cffa107d46d9cf98f0e3e813ef2ab98df5fc
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1809419
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
> Commit-Queue: Nico Weber <thakis@chromium.org>

Recipe-Nontrivial-Roll: chromiumos
Bug: 330461, 642711, 746763, 740301.
Change-Id: I5d09a93689e53e4339bff7b1173df938aee73a4d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1810400
Commit-Queue: Nico Weber <thakis@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
6 years ago
Edward Lesmes c6be56eced Revert "Set core.trustctime=false during git checkout operations."
This reverts commit 3f79763629.

Reason for revert: 
Breaks Skia https://skia-review.googlesource.com/c/buildbot/+/242590

Original change's description:
> Set core.trustctime=false during git checkout operations.
> 
> This should be faster and more general than running
> `git index --refresh` over all git repositories we can find.
> 
> See "Proposal: Setting core.trustctime=false on Chromium's bots"
> on infra-dev (linked to from bug 330461 comment 45).
> 
> Bug: 330461,642711,746763,740301.
> Change-Id: I7c35cffa107d46d9cf98f0e3e813ef2ab98df5fc
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1809419
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
> Commit-Queue: Nico Weber <thakis@chromium.org>

TBR=thakis@chromium.org,tandrii@google.com

Change-Id: I908ce92748f7b516efed5f2c02138ecfee306c88
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 330461, 642711, 746763, 740301.
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1811537
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Nico Weber <thakis@chromium.org>
6 years ago
Nico Weber 3f79763629 Set core.trustctime=false during git checkout operations.
This should be faster and more general than running
`git index --refresh` over all git repositories we can find.

See "Proposal: Setting core.trustctime=false on Chromium's bots"
on infra-dev (linked to from bug 330461 comment 45).

Bug: 330461,642711,746763,740301.
Change-Id: I7c35cffa107d46d9cf98f0e3e813ef2ab98df5fc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1809419
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Nico Weber <thakis@chromium.org>
6 years ago
recipe-roller 6bb29da829 Roll recipe dependencies (nontrivial).
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


Please review the expectation changes, and LGTM+CQ.


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/e7fec5ccb9e70bf17b7878e63052a874afa06856 Change is_luci default to True. (mmoss@google.com)


R=recipe-roller@chromium.org, tandrii@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I9eb8913a8f65b8f107e5856c46aceda10e14c00c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1752476
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
6 years ago
James Darpinian f994d87e56 Revert "Reland "depot_tools: Move six to depot_tools' .vpython""
This reverts commit d52b306f84.

Reason for revert: Broke ANGLE CQ with "ImportError: No module named six"
Example: https://ci.chromium.org/p/angle/builders/try/mac-dbg/6758

Original change's description:
> Reland "depot_tools: Move six to depot_tools' .vpython"
> 
> This is a reland of 3a98df0bdd
> 
> Original change's description:
> > depot_tools: Move six to depot_tools' .vpython
> > 
> > Bug: 984182
> > Change-Id: I34a242c8607624beb0f06a27c563e413dec99790
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1717492
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> > Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
> > Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> > Reviewed-by: Michael Moss <mmoss@chromium.org>
> 
> Bug: 984182
> Change-Id: Ibf7034201e65a18f833cf1e7432f81a903a99aae
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1732880
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

TBR=iannucci@chromium.org,mmoss@chromium.org,tandrii@chromium.org,ehmaldonado@chromium.org

Change-Id: Id4488249b607441df9f00aa51f4a1f2bc4ad5cfd
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 984182
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1739582
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Kenneth Russell <kbr@chromium.org>
6 years ago
Edward Lemur d52b306f84 Reland "depot_tools: Move six to depot_tools' .vpython"
This is a reland of 3a98df0bdd

Original change's description:
> depot_tools: Move six to depot_tools' .vpython
> 
> Bug: 984182
> Change-Id: I34a242c8607624beb0f06a27c563e413dec99790
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1717492
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Reviewed-by: Michael Moss <mmoss@chromium.org>

Bug: 984182
Change-Id: Ibf7034201e65a18f833cf1e7432f81a903a99aae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1732880
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Tamer Tas c5656169e4 Add optional target selection for resolving fixed revisions
{_resolve_fixed_revisions} resolves every revision property. After a test
failure, we don't want to resolve every revision always.

This CL allows updating a subset of revisions for a new checkout.

R=machenbach@chromium.org
TBR=tandrii@chromium.org

Bug: 950731, 982711
Change-Id: I3cdc5cdb882009886b82557427fdf9b02e860b39
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1720929
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Auto-Submit: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
6 years ago
Edward Lemur 4a8e453fbd bot_update: Disable metrics collection for bots, since it's not supported.
Change-Id: Iee0b783d725f15fbe3ab3d41ce2303a2ab1807af
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1713218
Reviewed-by: Andy Perelson <ajp@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
John Budorick 075cb05bde tryserver: add a 10 minute timeout on fetching CL info from gerrit.
Follow-up to crbug.com/970293.

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: skia
Change-Id: I2e0f71bc663e0ab571c0b9c98d57b2784366fb18
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1643679
Commit-Queue: John Budorick <jbudorick@chromium.org>
Auto-Submit: John Budorick <jbudorick@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
6 years ago
recipe-roller b712f77706 Roll recipe dependencies (nontrivial).
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


Please review the expectation changes, and LGTM+CQ.


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/b5922385482112ce31df891af71854c81662d69e Remove jsonResult defaulting to null (martiniss@google.com)


R=recipe-roller@chromium.org, tandrii@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I9791f5b80afe6529a6321eccbd4ae7d1a3f18a81
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1638960
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
6 years ago
Robert Iannucci 778c7f117a Make tryserver API only update the presentation of its own steps.
To sanely support concurrency, I need to make `api.step.nest` strictly
scoped; i.e. once you leave the nest context it closes all the steps
within that context.

This changes some wild 'active_step' uses to generate a step for the
module's own purpose.

R=martiniss@chromium.org, nodir@chromium.org, tandrii@chromium.org

Recipe-Nontrivial-Roll: build
Bug: 910369
Change-Id: I19aef31118faf9d3d078a2c5a0ebd48a4bfebd43
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1630044
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
6 years ago
Edward Lemur 99fe071354 Reland "bot_update: Pass target-ref to gclient."
This is a reland of f61af56950

crrev.com/c/1592577 should fix the issues with the original change.

Original change's description:
> bot_update: Pass target-ref to gclient.
>
> Bug: 956807
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: skia
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Recipe-Nontrivial-Roll: build
> Change-Id: I4d786d989446b8c1e3f39ef78c3c99cfe70808c0
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1588502
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

Bug: 956807
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: build
Change-Id: I16c22cb6aeb8e9dc1f55d77b1f12987720da3480
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1613341
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
6 years ago
recipe-roller 702766069b Roll recipe dependencies (nontrivial).
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


Please review the expectation changes, and LGTM+CQ.


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/125d102e429465dc3468b4b235f9e3a5df4b252c Reland: Refactor recipe engine step execution. (iannucci@chromium.org)


R=recipe-roller@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I7736e6b20f8a7ec9d5cf9eeb21456a1d0b37f189
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1609964
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
6 years ago
recipe-roller 66eeedc2ba Roll recipe dependencies (nontrivial).
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


Please review the expectation changes, and LGTM+CQ.


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/9b3ed1d5e912b8e9195060c3401da9b5370f2fce Revert "Refactor recipe engine step execution." (iannucci@chromium.org)


R=recipe-roller@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I628d01d7a259024f677ca8fb5de7317ae9aa591f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1609960
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
6 years ago
recipe-roller 417f2332fc Roll recipe dependencies (nontrivial).
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


Please review the expectation changes, and LGTM+CQ.


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/440a45cae6b329192f60313e41714bd78e886c8d Refactor recipe engine step execution. (iannucci@chromium.org)


R=recipe-roller@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I6e85bbfdf2eb61a433a505f81db6b805b9231402
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1610858
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
6 years ago