Commit Graph

418 Commits (eb1bd62b919ac1f7e5e3fdbdc8d6da232111baf1)

Author SHA1 Message Date
Edward Lesmes eb1bd62b91 presubmit_support: Add flags for Gerrit project and target branch.
Will be used in a follow-up CL to initialize code-owners client.

Recipe-Nontrivial-Roll: build
Change-Id: Iefe9176320b4d1ae7715e88a8db132e815be76ba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2717979
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
4 years ago
Stephen Martinis fb09de29ea Revert "Use GetCodeOwnersClient in presubmit_support"
This reverts commit 7a67bca65a.

Reason for revert: Probably broke chromium presubmit, see https://ci.chromium.org/ui/p/chromium/builders/try/chromium_presubmit/1152280/overview

Original change's description:
> Use GetCodeOwnersClient in presubmit_support
>
> This change also adds a target_ref flag to presubmit_support.py.
>
> Recipe-Nontrivial-Roll: build
> Change-Id: I6de6bb87fc1482b88d9fbebe5e4ad1dbd8ce9748
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2702792
> Commit-Queue: Gavin Mak <gavinmak@google.com>
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>

Change-Id: I2cca469f14194f65306baab7928ddddd48033a3b
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2718691
Auto-Submit: Stephen Martinis <martiniss@chromium.org>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
4 years ago
Gavin Mak 7a67bca65a Use GetCodeOwnersClient in presubmit_support
This change also adds a target_ref flag to presubmit_support.py.

Recipe-Nontrivial-Roll: build
Change-Id: I6de6bb87fc1482b88d9fbebe5e4ad1dbd8ce9748
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2702792
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
4 years ago
Jacob Kopczynski e71d0282d1 Add list of canonical footers
Includes all footers used by the recipe. Other footers used only by presubmit
hooks not included.

Bug: chromium:1079219
Change-Id: I9c351bda158bc7c1e64a8254feb7055dc31fb6db
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2707709
Auto-Submit: Jacob Kopczynski <jkop@chromium.org>
Commit-Queue: Jacob Kopczynski <jkop@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
4 years ago
Jacob Kopczynski 03918630fa Refactor get_footers
It will be easier to support centralized management of what footers exist if we
require all callers to specifically declare which footers they want to check.
This is a partial step towards allowing that without increasing the number of
queries required. It introduced cache staleness as a potential issue, but should
not matter in practice.

Bug: chromium:1079219
Change-Id: I678fd366202bac5a4efa4258ffe02e70c64589bb
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: chromiumos
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2664790
Commit-Queue: Jacob Kopczynski <jkop@chromium.org>
Auto-Submit: Jacob Kopczynski <jkop@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
4 years ago
Henrique Nakashima fa32fc9cc9 Revert "[recipe_modules/depot_tools] Add dirmd_path to api"
This reverts commit 25b2cb499a.

Reason for revert: Used copy of dirmd in third_party/ instead,
making this unnecessary.

Original change's description:
> [recipe_modules/depot_tools] Add dirmd_path to api
>
> This is required to parse DIR_METADATA files.
>
> Bug: 1135347
> Change-Id: I620656d4dbec5e918bfc0f535397869c869446b8
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2644827
> Commit-Queue: Henrique Nakashima <hnakashima@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>

TBR=tandrii@google.com,hnakashima@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: 1135347
Change-Id: I8ea8dfe8f3587177493fb37a4a37eb24266240e8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2686631
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
4 years ago
Henrique Nakashima 25b2cb499a [recipe_modules/depot_tools] Add dirmd_path to api
This is required to parse DIR_METADATA files.

Bug: 1135347
Change-Id: I620656d4dbec5e918bfc0f535397869c869446b8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2644827
Commit-Queue: Henrique Nakashima <hnakashima@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
4 years ago
Josip Sokcevic 2c9dbb89fb Revert "Add previous_got_revision and previous_got_revision_cp to output"
This reverts commit 6d0c0ffef5.

Reason for revert: this appears to  break builders if reference
is a commit hash:

git log --format=%B -n1 HEAD failed with code 128

Original change's description:
> Add previous_got_revision and previous_got_revision_cp to output
>
> As chrome browser infra works to improve CQ times, we would like to
> easily glance at builds and see how many new commits were checked out.
> Currently you have to open up bot_update stdout to see the previous
> HEAD revision and go from there. Having this conveniently outputted
> would be very convenient.
>
> If the bot has previously checked out the repo, output which revision
> and commit position were checked out.
>
> example LED build showing new output: https://ci.chromium.org/swarming/task/51592e46d6a70e10?server=chromium-swarm.appspot.com
>
> Change-Id: Ib8a6f152a593446ce4082477bc9ada44f1fd1fc3
> Bug: 1170568
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2644829
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>

TBR=infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com,kimstephanie@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: 1170568
Change-Id: I50d6ef6802c2eb1614cca804391c7826a79a2cf9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2657023
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Stephanie Kim 6d0c0ffef5 Add previous_got_revision and previous_got_revision_cp to output
As chrome browser infra works to improve CQ times, we would like to
easily glance at builds and see how many new commits were checked out.
Currently you have to open up bot_update stdout to see the previous
HEAD revision and go from there. Having this conveniently outputted
would be very convenient.

If the bot has previously checked out the repo, output which revision
and commit position were checked out.

example LED build showing new output: https://ci.chromium.org/swarming/task/51592e46d6a70e10?server=chromium-swarm.appspot.com

Change-Id: Ib8a6f152a593446ce4082477bc9ada44f1fd1fc3
Bug: 1170568
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2644829
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
4 years ago
Josip Sokcevic 76d0e6aca7 [bot_update] Resolve revision only if patching
If bot_update is not patching, it shouldn't look into gitiles patch
information.

R=ehmaldonado@google.com

Bug: 1147625
Recipe-Nontrivial-Roll: build
Change-Id: I111f0dc1c36d588341c02930bf1cda1f5b8cc2e7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2644199
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Edward Lesmes d686b1d1c8 bot_update: Set encrypted debug headers when retrying fetch.
Bug: b/178502800
Bug: 1170806
Change-Id: If66cf06b9954520ddd909dce869e2a15da3055ea
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2650965
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Josip Sokcevic 361ccce124 Revert "Support main branch in bot_update"
This reverts commit e7a31f42cd.

Reason for revert: has typo and doesn't work if typo is fixed

Original change's description:
> Support main branch in bot_update
>
> Change-Id: I58e88e9505779c309ac0af9d5b78b45feaa75543
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528669
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>

TBR=ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Change-Id: Ib6b266250b854e2554249b956766214be84ec522
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2643961
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Josip Sokcevic 60b433cf0b Enable packfile offloading on all builders
R=ehmaldonado@chromium.org

Bug: 1136986
Change-Id: I012f0d29589460b8cc7809c9c011811dff476a02
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2616898
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Garrett Beaty e856b6bba8 Enable set_output_commit to be used with RevisionFallbackChain.
When set_output_commit is True and the repo associated with the
got_revision property does not use commit positions, buiders whose
gclient config uses RevisionFallbackChain for that repo will raise an
exception when attempting set the output commit because a string is
expected for the revision value, not a RevisionFallbackChain object.
This changes the set_output_commit code to resolve the revision so that
the output commit can be set in those cases.

Bug: 1159749, 1141886
Change-Id: Iddf1d49871028d9500eefb0eed07e36ce7e6b0d6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2612223
Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Josip Sokcevic 7e73512268 Revert "Use HEAD instead of hardcoded branch"
This reverts commit 6651fd8847.

Reason for revert: broke some cros builds (check original CL for
discussion)

Original change's description:
> Use HEAD instead of hardcoded branch
>
> Repositories may have different default branches. This ensures to fetch
> remote HEAD branch which is default branch. Only downside to this
> approach is that actual branch (e.g. origin/main) reference won't be
> updated. Since we don't need it for this particular case, it should be
> fine to use HEAD.
>
> R=​apolito@google.com, ehmaldonaldo@chromium.org
>
> Bug: 1155492
> Recipe-Nontrivial-Roll: chromiumos
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Change-Id: Ib2e6e50901a0a10ed647acea3aeb517b83567702
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2580190
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>

TBR=ehmaldonado@chromium.org,apolito@google.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: 1155492
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: build
Change-Id: Ibd9f67bea71ee4a644b1077a365dbcd2a2c469f0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2597065
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
4 years ago
Josip Sokcevic d8fe52139f Check PACKFILE_OFFLOADING env variable
If set, set git config fetch.uriprotocols to https. This will be
initially set by codesearch builders. Once tested, this env variable can
be removed.

R=ehmaldonaldo@chromium.org

Bug: 1136986
Change-Id: I1c3da87bdbc74d18585ed00ffdc2cf21b84b16fa
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2591808
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Josip Sokcevic 6651fd8847 Use HEAD instead of hardcoded branch
Repositories may have different default branches. This ensures to fetch
remote HEAD branch which is default branch. Only downside to this
approach is that actual branch (e.g. origin/main) reference won't be
updated. Since we don't need it for this particular case, it should be
fine to use HEAD.

R=apolito@google.com, ehmaldonaldo@chromium.org

Bug: 1155492
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Change-Id: Ib2e6e50901a0a10ed647acea3aeb517b83567702
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2580190
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
4 years ago
Nodir Turakulov 260eb0f662 [tryserver] Report affected files via property
To enable aggregate analysis on changed files in BigQuery, conditionally
report them via an output property.

This CL causes a non-trivial roll only because of the sorting.

Recipe-Nontrivial-Roll: build
Bug: 1151655
Change-Id: Ie7a6c622196143c1aef07af9a9a356fb4202ea56
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2553349
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
4 years ago
Josip Sokcevic 9c0dc3026e Replace non-inclusive terms
Bug: 1118410, 1118413, 1118422, 1118424, 1118426
Bug: 1118428, 1118430, 1118434, 1118438, 1118442
Bug: 1118453, 1118454
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: build
Change-Id: I43e6305f8c3136774f01b57f12bb442bcb863371
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2551388
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
4 years ago
Xinan Lin 26ced49089 Improve the readability on variable name added in crrev/c/2528670
Bug: 1146487
Change-Id: I60476ac60ede4810f63403791386128508631db3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2551386
Commit-Queue: Xinan Lin <linxinan@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Xinan Lin <linxinan@chromium.org>
4 years ago
Xinan Lin 5589652282 Revert "Improve the setup logic for tryserver"
This reverts commit 4447336100.

Reason for revert: Breaks infra repo. See crrev.com/c/2548285

Original change's description:
> Improve the setup logic for tryserver
>
> If the given change does not have a valid gerrit host name, stop
> initializing the tryserver module.
> Also polished the variable name and docstr added in crrev/c/2528670.
>
> Recipe-Nontrivial-Roll: infra
> Bug:1146487
> Change-Id: I38193b25d37c724fd41d898eb020237fbf7cecdf
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2544000
> Commit-Queue: Xinan Lin <linxinan@chromium.org>
> Reviewed-by: Nodir Turakulov <nodir@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>

TBR=nodir@chromium.org,tandrii@google.com,recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com,linxinan@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

Change-Id: I3871866454adab816ab5549f8273fe16af2d9adb
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1146487
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2548409
Reviewed-by: Xinan Lin <linxinan@chromium.org>
Commit-Queue: Xinan Lin <linxinan@chromium.org>
5 years ago
Xinan Lin 4447336100 Improve the setup logic for tryserver
If the given change does not have a valid gerrit host name, stop
initializing the tryserver module.
Also polished the variable name and docstr added in crrev/c/2528670.

Recipe-Nontrivial-Roll: infra
Bug:1146487
Change-Id: I38193b25d37c724fd41d898eb020237fbf7cecdf
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2544000
Commit-Queue: Xinan Lin <linxinan@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
5 years ago
Xinan Lin da73999f79 Make bot_update and tryserver module not crash for multiple changes
Currently bot_update and tryserver crashes at initialization step if
the input contains multiple gerrit changes. This is not friendly for
CrOS CLs, e.g. crrev.com/c/2536761.

This CL moves the “crash” out of initialization, thus the consumer
of these modules could control which gerrit change to pass to bot_update
and tryserver.

This CL does NOT enable the modules here to support multiple patchset,
but allows consumers to choose one.

Bug:1146487
Change-Id: Iec46a8041189912e34da9bc5e8ff4611123ec9e0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528670
Commit-Queue: Xinan Lin <linxinan@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
5 years ago
Edward Lesmes 17cd53de67 Revert "Return HEAD when ref is refs/heads/main"
This reverts commit cbdf0c1e41.

Reason for revert: 
Broke tint bots

From log:
https://logs.chromium.org/logs/tint/buildbucket/cr-buildbucket.appspot.com/8864003131025797600/+/steps/bot_update/0/stdout 

Bot page:
https://ci.chromium.org/p/tint/builders/try/linux-clang-rel-x64/b8864003131025797600?

Original change's description:
> Return HEAD when ref is refs/heads/main
>
> R=​ehmaldonado@chromium.org
>
> Change-Id: I67b40cf48f0f9880e0f055a83d768bbefd704892
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528118
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
> Reviewed-by: Dirk Pranke <dpranke@google.com>

TBR=dpranke@google.com,ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

Bug: 1147494
Change-Id: Ieb21e15161ccc267f5d453a0196b1f0a2b43f366
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2529700
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Josip Sokcevic cbdf0c1e41 Return HEAD when ref is refs/heads/main
R=ehmaldonado@chromium.org

Change-Id: I67b40cf48f0f9880e0f055a83d768bbefd704892
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528118
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
5 years ago
Josip Sokcevic e7a31f42cd Support main branch in bot_update
Change-Id: I58e88e9505779c309ac0af9d5b78b45feaa75543
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2528669
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
Josip Sokcevic 982b2a7570 Add move change to Gerrit recipes
Will be used by a LUCI cron to move Gerrit changes for repos that are in
transition state #1 and #2.

R=ehmaldonado@chromium.org

Change-Id: Id7383ef677aa1cc90102c9a2aadbcc17712e150d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2522559
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Quinten Yearsley b674f8a277 Edit depot_tools recipe_modules docstrings for consistency
Change-Id: I454e9c90d3159598ae34d97d22c96b9d20d55f80
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2521656
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
5 years ago
Jeff Yoon 6fadc23db4 Adding OWNERS file for osx_sdk recipe module
Updating the OWNERS for osx sdk packages. List suggested from
crrev/c/2422449.

Change-Id: Id582737a539d8c10ec3437994068c33dd8c6d46a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2424556
Commit-Queue: Jeff Yoon <jeffyoon@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Reviewed-by: Gary Tong <gatong@chromium.org>
5 years ago
Stephanie Kim da6894f7a9 Add enforce_fetch to ensure_checkout cmd arg
The enforce_fetch kwarg was being ignored. This change adds the
--enforce_fetch to the resources/bot_update cmd.
Confirmed here through led: b852fa5e4bb77d6fb2aa69ec87ee57b5d2/+/steps/bot_update/0/logs/execution_details/0

Bug: 1129235
Change-Id: I78c5d62841fda6dafbb709301e26c5047df2d58d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2444114
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim b677e98529 Reland "Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache""
This is a reland of 3774c589cb

Added missing options.enforce_fetch arg

Confirmed via a fake test roll with this CL's patch commit that the
bot_update step passes with this change:
https://chromium-review.googlesource.com/c/chromium/tools/build/+/2443649

Original change's description:
> Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache"
>
> This is a reland of f38f54f2f1
>
> Original change's description:
> > Add optional enforce_fetch kwarg to fetch and refresh git cache
> >
> > Allows builders to do a force refresh of the git cache to ensure a new
> > fetch happens. android-binary-size needs this to get an accurate diff
> > of the passed in src revision and the applied patch:
> > https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
> >
> > Bug: 1129235
> > Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> > Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
>
> Bug: 1129235
> Change-Id: I034a91a8b57821d08b10042f7fc94d32350540ba
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443235
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>

Bug: 1129235
Change-Id: Ibbd4535f8529243d2dc1d4d32881d8a61efe374a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443574
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim d628ffbd8f Revert "Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache""
This reverts commit 3774c589cb.

Reason for revert: ea31a9d6c2771701fcaa11b47a2d735c19/+/steps/bot_update/0/stdout Going to revert and test with led before landing another "fix"

Original change's description:
> Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache"
>
> This is a reland of f38f54f2f1
>
> Original change's description:
> > Add optional enforce_fetch kwarg to fetch and refresh git cache
> >
> > Allows builders to do a force refresh of the git cache to ensure a new
> > fetch happens. android-binary-size needs this to get an accurate diff
> > of the passed in src revision and the applied patch:
> > https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
> >
> > Bug: 1129235
> > Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> > Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
>
> Bug: 1129235
> Change-Id: I034a91a8b57821d08b10042f7fc94d32350540ba
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443235
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>

TBR=tandrii@google.com,ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com,kimstephanie@google.com

Change-Id: I305215eed4128b13c9b5b524f4cbf3203037d582
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1129235
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443573
Reviewed-by: Stephanie Kim <kimstephanie@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim 3774c589cb Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache"
This is a reland of f38f54f2f1

Original change's description:
> Add optional enforce_fetch kwarg to fetch and refresh git cache
>
> Allows builders to do a force refresh of the git cache to ensure a new
> fetch happens. android-binary-size needs this to get an accurate diff
> of the passed in src revision and the applied patch:
> https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
>
> Bug: 1129235
> Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>

Bug: 1129235
Change-Id: I034a91a8b57821d08b10042f7fc94d32350540ba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443235
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim 98218e9219 Revert "Add optional enforce_fetch kwarg to fetch and refresh git cache"
This reverts commit f38f54f2f1.

Reason for revert: Seeing errors about missing args 007f2e16445872d9237cec0f72bb239fd6/+/steps/bot_update/0/stdout

Original change's description:
> Add optional enforce_fetch kwarg to fetch and refresh git cache
>
> Allows builders to do a force refresh of the git cache to ensure a new
> fetch happens. android-binary-size needs this to get an accurate diff
> of the passed in src revision and the applied patch:
> https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
>
> Bug: 1129235
> Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>

TBR=tandrii@google.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com,kimstephanie@google.com

Change-Id: Ib867742c624f085d3dd56422c2d63c15fdc84f06
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1129235
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443231
Reviewed-by: Stephanie Kim <kimstephanie@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
5 years ago
Stephanie Kim f38f54f2f1 Add optional enforce_fetch kwarg to fetch and refresh git cache
Allows builders to do a force refresh of the git cache to ensure a new
fetch happens. android-binary-size needs this to get an accurate diff
of the passed in src revision and the applied patch:
https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19

Bug: 1129235
Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
5 years ago
Jeff Yoon d1580d53b7 Add xcode version for Mac OS 10.15.4 in osx_sdk
Adding a version map entry for 10.15.6 to default to XCode 12
GM (12a7209) as machines upgrade to 10.15.

Bug: 1128042, 1128044
Change-Id: I6396b67fdce006d19be35d972602bac3bd4ae0a7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2422449
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Jeff Yoon <jeffyoon@chromium.org>
5 years ago
Takuto Ikuta 5c4c28f111 [recipes] remove is_luci specification
This is always True now.

Bug: 1128847
Change-Id: Ief951e9312ea8bd13c98b4bbc87c18d7307bb43a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2413667
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
5 years ago
Andrii Shyshkalov 7b7bdcd075 tandrii: switch OWNERS to @google.com account.
R=tikuta
TBR=tandrii@chromium.org

Change-Id: I8ce4eb78e8ec2cf4c6d816b8a5ca63dbb0de66c6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2411985
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
5 years ago
Garrett Beaty 71198cd4a2 Handle failures in the bot_update script.
Currently, if the bot_update script fails, then recipe code that makes
assumptions about the returned output JSON cause exceptions to be
raised. This fixes a couple instances in the code so that the bot_update
failure should surface instead of an uncaught exception.

Change-Id: I031e613841b03e5eb4f6d9ca7adb5085088a3077
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2405708
Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
5 years ago
recipe-roller 983a47d55a Roll recipe dependencies (nontrivial).
This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.

Please review the expectation changes, and LGTM+CQ.

recipe_engine:
0df5425581
  0df5425 (iannucci@chromium.org)
      [test] Filter stack traces by default.

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.

R=iannucci@chromium.org, martiniss@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I8d793d12c337bd2d04d07f386768bae0a2cb9497
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2403805
Auto-Submit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
5 years ago
Garrett Beaty 6c484875cd Ensure all keys in the bot_update revisions dict use forward slash.
On Windows, the path that gets computed from the gitiles commit uses
backslashes, which won't match the format of the paths that
bot_update.py uses.

Bug: 1047893
Change-Id: Ie1f6c070558fe00d6f4b1f2fd453e430b21a55ed
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2380590
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
5 years ago
Saagar Sanghavi 0fa91d0f35 ResultDB wrapping for CQ/CI builds.
Add ('rdb', 'stream', '--') to wrapper.
Uses -- to send args to subprocess. (This was the bug with original CL)

Note - to test this, we will have to push changes and revert
immediately if we notice any breakage.

Recipe-Nontrivial-Roll: build
Change-Id: Ibe46314c2f791a99c0e670ff6cc44f6f24fb9f70
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2342254
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Erik Staab <estaab@chromium.org>
Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
5 years ago
Ben Pastene fb9c1110ef Revert "ResultDB reporting for bot-triggered builds"
This reverts commit bd65053490.

Reason for revert: "flag provided but not defined: -u"
https://crbug.com/1113350

Original change's description:
> ResultDB reporting for bot-triggered builds
> 
> Recipe-Nontrivial-Roll: build
> Change-Id: Id062a66c8a37aeedc85eaa16b1aa1eea4304f5c8
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2330655
> Reviewed-by: Garrett Beaty <gbeaty@chromium.org>
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Nodir Turakulov <nodir@chromium.org>
> Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>

TBR=nodir@chromium.org,estaab@chromium.org,ehmaldonado@chromium.org,gbeaty@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,saagarsanghavi@google.com

Bug: 1113350
Change-Id: I2c496d6c170a2ffb9226c5bb201d467f5847de6a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2340035
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
5 years ago
Saagar Sanghavi bd65053490 ResultDB reporting for bot-triggered builds
Recipe-Nontrivial-Roll: build
Change-Id: Id062a66c8a37aeedc85eaa16b1aa1eea4304f5c8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2330655
Reviewed-by: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
5 years ago
Garrett Beaty 3e5d86a237 Generate revisions in json output for refs/ and origin/.
Right now the default test data for bot_update doesn't transform
revisions specifying refs for the revision except for refs starting with
refs/ on the first solution. This means that the output will just have
the ref name as the revision value, which isn't compatible with code
that requires an actual revision value.

Bug: 1047893
Change-Id: I016ad5e8db5a293dd5e6dacb5b8beba51e000949
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: chrome_release
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2325761
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Greg Guterman 71fd8abb9d Add explicit "python" to script for Windows compatibility
Remove cwd augmentation for good measure. This should be done by the
caller.

Bug: 923016
Change-Id: I7f6618d7a9e56f70168fc71046412c6d462f40fb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2311320
Auto-Submit: Gregory Guterman <guterman@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Edward Lesmes e4e1fb30c8 bot_update: Map refs to remote refs and don't checkout after clone.
Map refs to remote refs when fetching refs from origin
(e.g. refs/heads/master -> refs/remotes/origin/master)
so that a detached HEAD is not necessary.

Bug: 1104182
Change-Id: Icb751505de1bf443e5d45c4b37425223dbf6e088
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2310849
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Garrett Beaty 6081b23b29 Modify bot_update to support setting blamelist pins.
This adds a parameter to bot_update.ensure_checkout that will cause
blamelist pins to be set for all of the repos whose revision is not
determined by DEPS files: the repo for the gitiles input, the repo for
each solution in the gclient config and any repos with the revision set
by the revisions field of the gclient config.

Bug: 1047893
Change-Id: Ie3eee41dbf15be014185b6f255ee3d4cfd40b4e4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2293514
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Garrett Beaty c8c88f9c55 Use fixed revisions in bot_update's test API output_json.
Currently the output_json method generates revisions for projects that
are in the revision_mapping and only those projects appear in the
manifest and source_manifest, using the generated revision. This method
is used to construct the default json output for bot_update in tests.
This has 2 problems:
1. If a project has a fixed revision, its manifest entries and the
property used the generated revision, meaning the gitiles input revision
will not be used in the json output.
2. If a project has a fixed revision and isn't present in
revision_mapping then manifest entries aren't generated for it.

This changes the method to use the revisions in the fixed_revisions dict
and to create manifest entries for the projects that appear in the
fixed_revisions dict which should make fewer situations where
bot_update's output_json needs to be explicitly set.

Change-Id: I99de52edc9248c28e9d3aa3a04cc77bf9fccbe37
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: chrome_release
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2304644
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
5 years ago
Edward Lesmes 720e010ba1 Revert "bot_update: Refactor branch to detach and checkout."
This reverts commit b99e61c58a.

Reason for revert:
Doesn't work for refs/branch-heads/*

Original change's description:
> bot_update: Refactor branch to detach and checkout.
> 
> Refactor _git_checkout to use information from
> get_target_branch_and_revision to know what branch to checkout after
> cloning.
> 
> Bug: 1104182
> Change-Id: Ib3ba57ca0b6803f172b85121c2a4b123f17bfb8c
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2303291
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

TBR=ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com

Change-Id: If51423df99fd9c164f8e42e0220ee2d9bc2a39f6
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1104182
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2310293
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago