Make autoninja path also available via depot_tools recipe module,
and make autoninja used where it is needed.
Bug: b/77176746
Change-Id: I8553cb3fb32f98e2e3ff420c255461b7e7da7df6
Reviewed-on: https://chromium-review.googlesource.com/c/1293112
Reviewed-by: Fumitoshi Ukai <ukai@chromium.org>
Reviewed-by: Shinya Kawanaka <shinyak@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Yoshisato Yanagisawa <yyanagisawa@chromium.org>
The LUCI environment alreadly does `git retry` style retries by virtue
of it's Go git-wrapper program. No need to put an extra python
interpreter in the middle.
R=hinoka@chromium.org, nodir@chromium.org
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Change-Id: I2918d7f413dde667fccd45c83ad6f4b96c2afe2e
Reviewed-on: https://chromium-review.googlesource.com/c/1292236
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Bug: chromium:892712
Change-Id: I68e1822a9a7205aaab3836be3f57a2f1feb1c9a6
Reviewed-on: https://chromium-review.googlesource.com/c/1289231
Commit-Queue: Scott Graham <scottmg@chromium.org>
Auto-Submit: Scott Graham <scottmg@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
This has been low-key driving me bananas for a while, and I've seen
it confuse sheriffs etc many times.
Change-Id: If46ab55821447a3a70a3f0ea774c017f29fc7400
Reviewed-on: https://chromium-review.googlesource.com/c/1286819
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
Some projects that use the git module (eg flutter) require tags.
Bug: 870558
Change-Id: Idae50ef5c8a092e5f0109b6c14c562268a2ad236
Reviewed-on: https://chromium-review.googlesource.com/c/1274245
Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
This should allow better integration with buildbucket named cache
support. Since this was already parameterized it shouldn't actually
break anything; previously the directory would be deleted on every run,
now it will still be deleted on every run (but we can add a buildbucket
named cache entry for it to preserve it).
R=tandrii@chromium.org, vadimsh@chromium.org
Bug: 877775
Change-Id: I1184fb59a3ed18bfda5fcb288903201b01527096
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Reviewed-on: https://chromium-review.googlesource.com/c/1263861
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
It has been True by default for a while, and there is no need to override it.
Bug: 891917
Change-Id: I2598a2230b0ea38a647a533757331c541b871971
Reviewed-on: https://chromium-review.googlesource.com/c/1260057
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
It looks like PNaCl actually depends on MSVC's presance, but doesn't
actually run any compile steps with it (instead using clang). That means
that mspdbsrv never actually runs during their build, and then the
taskkill fails.
R=hinoka@chromium.org, tandrii@chromium.org
Bug: 861512
Change-Id: I004d28f198224adaf16b1d3f14401b0d2a7d700b
Reviewed-on: https://chromium-review.googlesource.com/1232885
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
It is not used. get_gerrit_patch_root is used instead.
Bug: 694348
Change-Id: Icc717e38cbcaa8050c8ce8a364202b81d7f9cf2c
Reviewed-on: https://chromium-review.googlesource.com/1231282
Auto-Submit: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Use get_gerrit_patch_root in bot_update.ensure_checkout.
This fixes some tests expectations that expected CI builds to pass --patch_root
flag.
Also unify "root" and "patch_root" variables in ensure_checkout. Simpler.
This CL requires nontrivial rolls in all downstream packages because all
packages have incorrect expectations and/or tests.
In particular, this removes --patch_root flag from non-tryjob test
cases. Some tests are supposed to test tryjob scenario, but they use
properties.generic instead of properties.tryserver (or buildbucket.try_build)
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: release_scripts
Recipe-Nontrivial-Roll: skiabuildbot
Bug: 694348
Change-Id: Ibc594fa42e3b4d2823672b41dda2d97154b10d38
Reviewed-on: https://chromium-review.googlesource.com/1231280
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Update set_patch_project_revision to use repo_path_map instead of
patch_projects.
Also rename to set_patch_repo_revision.
Bug: 694348
Change-Id: Iba31f4910f8282160000b8fcef197052597530a4
Reviewed-on: https://chromium-review.googlesource.com/1226107
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Add a shortcut for getting local path to the patch repo.
In bot_update.api._destination_branch, start using it instead of reading
gclient config directly. This removes one more patch_project usage.
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Bug: 694348
Change-Id: If05371d0f6d11be6803ffa8392e029f5860dda25
Reviewed-on: https://chromium-review.googlesource.com/1217670
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Add tryserver.api.gerrit_change_info that returns detailed info about the
current Gerrit CL. It makes a Gerrit RPC in the background on the first read.
This step will be present in all tryjobs.
This mechanism can be used for extra details about the current CL.
This CL starts to use this in bot_update for gerrit_ref, and stops using
patch_ref property, and also update bot_update to use when the destination
branch is needed.
Also this CL removes no longer used properties in bot_update,
namely most of patch_* properties.
build.git manual roll: https://chromium-review.googlesource.com/c/chromium/tools/build/+/1217626
Recipe-Manual-Change: build
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: skiabuildbot
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: release_scripts
R=tandrii@chromium.org
Bug: 694348
Change-Id: I945d061b74278c3d1d227578abe16a64cdaca647
Reviewed-on: https://chromium-review.googlesource.com/1213557
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
CrOS chromium-pfq builder uses bot_update, but checks out Chromium on its own,
effectively ignoring input commit in bot_update.
To resolve a P0, add ignore_input_commit parameter.
Long term, bot_update API must be redesigned.
TBR=tandrii@chromium.org
Bug: 882094
Change-Id: I4f43348e3d5f81bf603dd2c110829bce58acad16
Reviewed-on: https://chromium-review.googlesource.com/1216787
Reviewed-by: Lann Martin <lannm@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
tryserver module's purpose is to provide high level API for patches.
Update it to use buildbucket.build.input.gerrit_changes[0].
Add helper properties for gerrit repo URL and gerrit ref.
This will be used by both bot_update and gclient.
R=hinoka@chromium.org, tandrii@chromium.org
Bug: 694348
Change-Id: I7eb416e75737310404c4354fa680fd3499eddaa7
Reviewed-on: https://chromium-review.googlesource.com/1211653
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Use self.m.buildbucket.build.input.gitiles_commit.id instead.
It encapsulates revision and parent_got_revision properties.
Also, honor host and project of the specified commit.
This enables overriding non-first gclient solution at the build-level
for arbitrary buildbucket builds.
Bug: 877161
Change-Id: I3f92f04ca84a1d30cbbb2670f62bea2bd9ee5932
Reviewed-on: https://chromium-review.googlesource.com/1200066
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Use repo_path_map in all places where patch_projects is currently used.
Add gclient.get_repo_path function to resolve path to a repo given its URL.
infra.git requires a non-trivial roll because of incorrect expectations.
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Bug: 877161
Change-Id: Iaea0a3f14609bf0c18d4ecdf8564153b8052584d
Reviewed-on: https://chromium-review.googlesource.com/1208363
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
When checking if buildbucket.build_input.gitiles_commit exists, use
HasField protobuf method, as opposed to truth check. A protobuf message
field is always true.
Bug: 877161
Change-Id: Ie4a7e22b985776441a74283b09646e8837c326f1
Reviewed-on: https://chromium-review.googlesource.com/1196930
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This is in order to use built-in presubit recipe.
R=maruel
Change-Id: I3e50af7dc3a9ad83f0f358285395794ea599a4e8
Reviewed-on: https://chromium-review.googlesource.com/1176650
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Also add shortcuts, so we don't have to look through all the branches
if the patch is based on a common branch (i.e. master, infra/config, lkgr)
Bug: 870279
Change-Id: I625a8481dccac9a475b096b926e6fab7efe676b0
Reviewed-on: https://chromium-review.googlesource.com/1161094
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).
Please review the expectation changes, and LGTM+CQ.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
https://crrev.com/bd5111dec043a367d69d85b81289e2207aa26f6e [cipd] Clean docs, pylint errors, add pkg-fetch, pkg-deploy. (iannucci@chromium.org)
R=tandrii@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I19d44529f7caa41c9577e57f6a9a35815627c4c3
Reviewed-on: https://chromium-review.googlesource.com/1170243
Commit-Queue: Recipe Roller <recipe-roller@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
When syncing remote branches, checkout origin/<branch> instead of
<branch>, since there might be a local branch with that name.
Bug: 859703
Change-Id: I9a49b940f53f75a97e5458fdcf578c3b52b26de2
Reviewed-on: https://chromium-review.googlesource.com/1128271
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Today, whenever LUCI scheduler sets gitiles_commit, a revision property
is also always set, thus this CL will have no effect on today's production.
The goal of CL is to pave the way to avoid this duplication. For starters,
one could start real writing tests with
+ api.buildbucket.ci_build(..., revision='sha1')
without the need to duplicate the same revision as a property. All
recipes which WAI with such input are thus ready to work w/o 'revision'
property. After that, we'll be able to stop the duplication in LUCI scheduler.
R=hinoka
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Change-Id: I1773b1c3d2a98a9dbd1cfdd9ac2b4fb99745ee99
Reviewed-on: https://chromium-review.googlesource.com/1123713
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
It is not possible to use cwd argument, since none of the methods called by this
module define cwd explicitly and low-level modules like api.step do not support
it anymore, resulting in a TypeError exception thrown due to unexpected keyword
argument, e.g. see https://ci.chromium.org/swarming/task/3e74a52905a71110. Given
that any users using cwd argument would be broken anyway, it should be safe to
make this change.
Planned usage: https://crrev.com/c/1122874
R=tandrii@chromium.org, iannucci@chromium.org
Bug: 793687
Change-Id: Iadf6bf91d8d41dbc6751a848bb69d7922a206ade
Reviewed-on: https://chromium-review.googlesource.com/1122873
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
gclient should be the one syncing the solutions to the appropriate
revisions.
Bug: 850812, 853032
Change-Id: Ieefc5661627d4864deb0d4e7053168a99da29d29
Reviewed-on: https://chromium-review.googlesource.com/1102833
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
shallow mode is a special git mode that only checks out a shallow copy
of git so the full history is not downloaded. It is not well supported
by googlesource.com servers because git has to do a giant negotiation
with the server and transmit every single commit that the local checkout
has or doesn't have (instead of just the latest commit of each ref),
which ends up taking more than 10x longer than a non-shallow checkout.
R=ehmaldonado, hinoka, iannucci
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Bug: 855137
Change-Id: I5f9e31f8b2730cf80b79bfd0e08201c33eb942f5
Reviewed-on: https://chromium-review.googlesource.com/1110450
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Allows easily getting at more than just the commit message.
I intend to look at the author to whitelist bots from binary size
checks (initially).
Bug: 702625
Change-Id: I684fdfacc88635ec4849216c74183f63f9d54810
Reviewed-on: https://chromium-review.googlesource.com/1070483
Commit-Queue: agrieve <agrieve@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
It fails when both the revision and the url are None.
Bug: 841936
Change-Id: Idef45a015624a92226d4ccd38ed5b978bf786993
Reviewed-on: https://chromium-review.googlesource.com/1053996
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
This is a reland of 3806b7fbd0
Original change's description:
> bot_update: Fetch references to be synced.
>
> If a solution is to be synced to a reference, such as
> refs/branch-heads/#### fetch it so that it can later be
> checked out.
>
> I tried this change on a LUCI task that runs chromium_presubmit
> for crrev.com/c/1048868. The bot_update works as expected:
> https://ci.chromium.org/swarming/task/3d5b294cff467710?server=chromium-swarm.appspot.com
>
>
> Bug: 840445
>
> Change-Id: I6444c0feb679ca89b2f4065215de915494e5082a
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Reviewed-on: https://chromium-review.googlesource.com/1049007
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Bug: 840445
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Change-Id: I64a97aafe5fab9a26c478262e9ca104dc59e61f5
Reviewed-on: https://chromium-review.googlesource.com/1052387
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
This reverts commit 3806b7fbd0.
Reason for revert: Broke downstream tot bots
Original change's description:
> bot_update: Fetch references to be synced.
>
> If a solution is to be synced to a reference, such as
> refs/branch-heads/#### fetch it so that it can later be
> checked out.
>
> I tried this change on a LUCI task that runs chromium_presubmit
> for crrev.com/c/1048868. The bot_update works as expected:
> https://ci.chromium.org/swarming/task/3d5b294cff467710?server=chromium-swarm.appspot.com
>
>
> Bug: 840445
>
> Change-Id: I6444c0feb679ca89b2f4065215de915494e5082a
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Reviewed-on: https://chromium-review.googlesource.com/1049007
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
TBR=mmoss@chromium.org,machenbach@chromium.org,tandrii@chromium.org,ehmaldonado@chromium.org
Change-Id: If94df8684e634504d8a0c11b2aa5862a458ba19f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 840445, 841306
Reviewed-on: https://chromium-review.googlesource.com/1052187
Reviewed-by: agrieve <agrieve@chromium.org>
Commit-Queue: agrieve <agrieve@chromium.org>
If a solution is to be synced to a reference, such as
refs/branch-heads/#### fetch it so that it can later be
checked out.
I tried this change on a LUCI task that runs chromium_presubmit
for crrev.com/c/1048868. The bot_update works as expected:
https://ci.chromium.org/swarming/task/3d5b294cff467710?server=chromium-swarm.appspot.com
Bug: 840445
Change-Id: I6444c0feb679ca89b2f4065215de915494e5082a
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Reviewed-on: https://chromium-review.googlesource.com/1049007
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Otherwise, the origin URLs for the checkout and the patch won't match
and applying the patch on gclient will fail.
Bug: 643346
Change-Id: If005bb10e579b3d50f9b6d0ff39440bb9a300cbd
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Reviewed-on: https://chromium-review.googlesource.com/1043254
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
This is a reland of e86fe982e8
Original change's description:
> bot_update: Set user.{name,email} when applying patch refs.
>
> git rebase complains when user name/email has not been configured, as was
> the case for the Chromium trybot.
>
> Bug: 643346
> Change-Id: Iab7dac0a98608e09b63a53d5bacb98925701fbfd
> Reviewed-on: https://chromium-review.googlesource.com/1031524
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Bug: 643346
Change-Id: I4b4d07675222d39d704560ed84f9c221e87aba8b
Reviewed-on: https://chromium-review.googlesource.com/1036309
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This is needed for release builders to checkout the release tags that
will be created when the buildspec branch/release info is migrated out
to the regular repos.
BUG=825063
R=jbudorick@google.com
Change-Id: I5f0e4cdd94de11c59f9b4a5c11aabea129e81523
Reviewed-on: https://chromium-review.googlesource.com/1036526
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Michael Moss <mmoss@chromium.org>
This reverts commit e86fe982e8.
Reason for revert: Probably causing outages on win7_chromium_rel_ng.
Original change's description:
> bot_update: Set user.{name,email} when applying patch refs.
>
> git rebase complains when user name/email has not been configured, as was
> the case for the Chromium trybot.
>
> Bug: 643346
> Change-Id: Iab7dac0a98608e09b63a53d5bacb98925701fbfd
> Reviewed-on: https://chromium-review.googlesource.com/1031524
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
TBR=agable@chromium.org,ehmaldonado@chromium.org
Change-Id: Ic57398323476fccef34f766270d760c3975630a6
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 643346
Reviewed-on: https://chromium-review.googlesource.com/1036308
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
git rebase complains when user name/email has not been configured, as was
the case for the Chromium trybot.
Bug: 643346
Change-Id: Iab7dac0a98608e09b63a53d5bacb98925701fbfd
Reviewed-on: https://chromium-review.googlesource.com/1031524
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Previously, anything not equal to HEAD meant pinned revision.
This backfired on values like "origin/master" which are meant as
equivalent to HEAD, but which bot_update tried to ensure exist in
checkout, which was refreshed several times under assumption that
this is eventual consistency of git servers.
Instead, this CL treats repo as pinned if and only if the treeish
of the repo is matching sha1 regex.
R=hinoka@chromium.org
Bug: 838292
Change-Id: I9cb46535f46b2898cf54900ede7960ba08770d77
Post-Scriptum: git bisect + Led is awesome, took only 8 manual steps.
Reviewed-on: https://chromium-review.googlesource.com/1035793
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
This reverts commit 355b11a31f.
Reason for revert:
Easier to do this another way that doesn't cause expectation changes,
so it doesn't require non-trivial rolls.
Original change's description:
> bot_update: Patch on gclient by default.
>
> See [4] for the design doc for this change.
>
> We're moving the patching logic from bot_update to gclient.
> This way we can test patches of dependencies that modify their DEPS
> files as well, for example when ANGLE uploads a patch that modifies
> DEPS files to be tested by a Chromium trybot.
>
> Chromium Linux CQ and Angle trybots are already running this way
> (see [1] and [2]) and a PSA has already been sent [3]
>
> [1] https://chromium.googlesource.com/chromium/src/+/master/infra/config/global/cr-buildbucket.cfg#214
> [2] https://chromium.googlesource.com/angle/angle/+/infra/config/cr-buildbucket.cfg#105
> [3] https://groups.google.com/a/google.com/forum/#!topic/chrome-infra/FVHN2kdUX4I
> [4] https://docs.google.com/document/d/1GSTw4ysm5AYjNR6vMA7NJUxLK2VN9IpOEp_v0u-wyOE/edit?ts=5ad78165#
>
> Bug: 643346
> Change-Id: Ic9a9dcfae86aca730d35fbb26c26b21135e05bf3
> Recipe-Nontrivial-Roll: skia
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Recipe-Nontrivial-Roll: skiabuildbot
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: release_scripts
> Reviewed-on: https://chromium-review.googlesource.com/1014356
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>
TBR=agable@chromium.org,ehmaldonado@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 643346
Change-Id: I0e694b0e398770b8bd2e5a3170e158a7e54c5a61
Reviewed-on: https://chromium-review.googlesource.com/1029310
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
We should apply the patch only when gerrit_repo and gerrit_ref are passed,
instead of using only the --apply-patch-on-gclient flag.
R=agable@chromium.org
Bug: 643346
Change-Id: I28547aed9daeefa2657a108e0bacbfd4fc9fa07e
Reviewed-on: https://chromium-review.googlesource.com/989667
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Add some other people to recipes/OWNERS
Change-Id: I554d1fd354e628d2e62629be7ce20e899364c4a9
Reviewed-on: https://chromium-review.googlesource.com/984972
Commit-Queue: Nico Weber <thakis@chromium.org>
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
This adds a --enable-gclient-experiment flags that tells bot_update.py
to skip applying the patch, and instead forward the flags to gclient.
Bug: 643346
Change-Id: Ia4275a126e6adba54dfcc894d224c50c166db90e
Reviewed-on: https://chromium-review.googlesource.com/962938
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
This method hasn't been supported for a long time now (and besides, its
only for Rietveld).
R=agable@chromium.org, tandrii@chromium.org
Recipe-Manual-Change: infra
Change-Id: Ie6e63834dca67962db29f2cb407950ed85db55a7
Reviewed-on: https://chromium-review.googlesource.com/957832
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Also, some lines went over 80chars, so I ran 'yapf --style chromium' which
formatted some unrelated things.
BUG=905285
R=dpranke
Change-Id: Iee5f46d88a6e9782612cc4f9e5a2cb72d62ab6af
Reviewed-on: https://chromium-review.googlesource.com/907736
Commit-Queue: Thomas Anderson <thomasanderson@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
This reverts commit fa573785df.
Reason for revert: broke CQ on branches.
Example: https://ci.chromium.org/p/chromium/builders/luci.chromium.try/chromium_presubmit/28220
Original change's description:
> bot_update: allow rebasing the patch onto an older revision.
>
> When applying a rebase, we normally go from an older base commit to a newer one.
>
> A ---- B ---- C ---- D -origin/master
> \
> E -branch
>
> In this case, `git rebase D` would certainly work as expected. However,
> writing `git rebase B` would NOT get us to the following state:
>
> A ---- B ---- C ---- D -origin/master
> \
> E' -branch
>
> In fact, it would have no effect.
>
> This article http://matthew-brett.github.io/pydagogue/rebase_without_tears.html
> explains the general invocation as
> > `git rebase --onto <graft-point> <exclude-from> <include-from>`
> > If you don’t specify --onto, <graft-point> defaults to <exclude-from>
>
> So what's happening is, by writing `git rebase B` we're rebasing onto B,
> excluding commits that are in B. Commit C is not "in" B so it is kept and we're
> back to the starting point.
>
> So I suggest to change the invocation to `git rebase --onto B origin/master`,
> which rebases onto B, excluding commits that are in origin/master. This works
> more generally and allows rebasing "backwards".
>
> Bug: None
> Change-Id: I68e4d805811530b585550bc75099354fef4e9c15
> Reviewed-on: https://chromium-review.googlesource.com/904004
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
> Commit-Queue: Oleh Prypin <oprypin@chromium.org>
TBR=iannucci@chromium.org,tandrii@chromium.org,oprypin@chromium.org
Change-Id: I7243641d84428bf4504ee798bf31ad0afbfd4865
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: None
Reviewed-on: https://chromium-review.googlesource.com/905862
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Goal: reduce number of git fetches from remote to the minimum possible.
Minimum possible is either:
* 0 if desired revision is pinned (!= HEAD) and already exists in cache
* 1 if desired revision is HEAD
* >= 2 if revision is pinned but we talk temporarily
out-of-date git server not yet having it.
This CL achieves the above.
No fetch:
https://ci.chromium.org/swarming/task/39dad20230c84110?server=chromium-swarm.appspot.com
bot_update takes <1min, but it used to take at least 2. The steps are now like this:
...
===Running git cat-file -e 33bf8a94dcd5e0abbdf83e4afaed24b5180e3eb2===
In directory: /b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src
===Succeeded in 0.0 mins===
===Running /b/swarming/w/ir/cipd_bin_packages/bin/python -u /b/swarming/w/ir/recipe-checkout-dir/depot_tools/git_cache.py exists --quiet --cache-dir /b/swarming/w/ir/cache/git https://chromium.googlesource.com/chromium/src.git===
In directory: /b/swarming/w/ir/cache/builder/linux
/b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src
===Succeeded in 0.0 mins===
===Running git remote set-url origin /b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src===
In directory: /b/swarming/w/ir/cache/builder/linux/src
===Succeeded in 0.0 mins===
===Running git fetch origin===
In directory: /b/swarming/w/ir/cache/builder/linux/src
From /b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src
003eedbfee47..5210107a20e8 master -> origin/master
===Succeeded in 0.0 mins===
===Running git checkout --force 33bf8a94dcd5e0abbdf83e4afaed24b5180e3eb2===
...
Fetch & success:
https://ci.chromium.org/swarming/task/39db022b874ffd10?server=chromium-swarm.appspot.com
Fetch & retry because missing rev:
https://ci.chromium.org/swarming/task/39db04ce8b4b8910?server=chromium-swarm.appspot.com
Fetch because rev=HEAD:
https://ci.chromium.org/swarming/task/39db0c0dd5ab5d10?server=chromium-swarm.appspot.com
This has already been adopted in gclient.
Bug:
Change-Id: Id99892b62719fdf3f7e6e59058986d1500384f8d
Reviewed-on: https://chromium-review.googlesource.com/771591
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
When applying a rebase, we normally go from an older base commit to a newer one.
A ---- B ---- C ---- D -origin/master
\
E -branch
In this case, `git rebase D` would certainly work as expected. However,
writing `git rebase B` would NOT get us to the following state:
A ---- B ---- C ---- D -origin/master
\
E' -branch
In fact, it would have no effect.
This article http://matthew-brett.github.io/pydagogue/rebase_without_tears.html
explains the general invocation as
> `git rebase --onto <graft-point> <exclude-from> <include-from>`
> If you don’t specify --onto, <graft-point> defaults to <exclude-from>
So what's happening is, by writing `git rebase B` we're rebasing onto B,
excluding commits that are in B. Commit C is not "in" B so it is kept and we're
back to the starting point.
So I suggest to change the invocation to `git rebase --onto B origin/master`,
which rebases onto B, excluding commits that are in origin/master. This works
more generally and allows rebasing "backwards".
Bug: None
Change-Id: I68e4d805811530b585550bc75099354fef4e9c15
Reviewed-on: https://chromium-review.googlesource.com/904004
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Oleh Prypin <oprypin@chromium.org>
This is needed for run_presubmit.py to work.
BUG=pdfium:971
Change-Id: Ib598fbfd6eab3c0b7628301eb3bd54c548a4885e
Reviewed-on: https://chromium-review.googlesource.com/898039
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Lei Zhang <thestig@chromium.org>
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).
Please review the expectation changes, and LGTM+CQ.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
https://crrev.com/77667b5ba38c4d205a743d04eb625ac068bdd78e Always use Gerrit for tryserver recipe tests (agable@chromium.org)
R=dnj@chromium.org, martiniss@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I2df24323974ae0d3b3b0524240b64be5740977c6
Reviewed-on: https://chromium-review.googlesource.com/836760
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
This introduces a pragma: no cover, which is unfortunate.
But this is a prerequisite for removing rietveld support
entirely from the recipe engine without having to do
manual recipe rolls. The entire block of uncovered code
will be removed shortly.
Bug: 770408
Change-Id: I37319f3ae70413adcda655c443325bccffe695d8
Reviewed-on: https://chromium-review.googlesource.com/834455
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
This partially reverts commit d51ed57edb.
Reason for revert:
New git client for windows was rolled including fix for slow `git fetch`.
I guess smaller pack limit causes frequent bootstrap taking 2~3 minutes longer than
the case it does not happen.
Let me see what happen if we increase pack limit 9 -> 30.
I will increase this to 50 if this won't cause regression again.
Original change's description:
> git_cache: lower max num of .pack files before re-bootstrap on Win.
>
> It used to be 50, I think ~9 gives best results for Chromium on Win:
> on golo VM, it takes <4 minutes to re-boostrap + git fetch small
> delta, assuming zipped git checkout for bootstrap is fresh (~1day).
>
> For other repos, which are significantly smaller, this change should
> have minor effect if at all.
>
> Test: I tested this using `led` tool on Win7 machines running LUCI
> stack extensively. For example,
>
> * https://ci.chromium.org/swarming/task/3a0102e8c8657410
> shows case with few .pack files, hence just 1 fetch
>
> * https://ci.chromium.org/swarming/task/3a010282f9fd8010
> shows case with 39 .pack files and so bootstrapping + fetch.
> If you look at prior tasks on the same VM, you'd find this:
> https://ci.chromium.org/swarming/task/39ffe843d01ed010
> which spent 8 minutes doing 1 incremental fetch with 39 .pack
> files.
>
> **Troopers/Sheriffs**: This change is safe to revert.
> However, beware that you should also at the same time revert the recipe
> roll of this CL to the repo, in which the failed builder's recipe is
> located, typically `chromium/tools/build`.
>
> Bug: 749709
> Change-Id: I18e2b63283100d466e9fb981a9094862463f6909
> Reviewed-on: https://chromium-review.googlesource.com/787174
> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
> Reviewed-by: Takuto Ikuta <tikuta@google.com>
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 749709
Change-Id: I3052abe4a9b53277a60c0791a85355e7a0bbdf8f
Reviewed-on: https://chromium-review.googlesource.com/823544
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Takuto Ikuta <tikuta@google.com>
This will be used by various things in build and build_internal which
"need" a depot_tools on PATH (though I suspect most of those should be
refactored to use the depot_tools in chromium/src.git).
R=tandrii@chromium.org
Bug: 789808
Change-Id: I5d0590ccdc5482bd8e3048ad46a39b5aedacbddb
Reviewed-on: https://chromium-review.googlesource.com/828465
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
This is a reland of 9219d35688
The original was reverted due to a typo (core,quotePath instead
of core.quotePath). This version is fixed.
Original change's description:
> Use core.quotePath=false when git is listing files
>
> This prevents git from putting quotes around some file names
> (those that have astral-plane characters) and not around others.
>
> R=maruel
>
> Bug: 792302
>
> Change-Id: I3b6a6b36c4720116de811b40177b59aa25c263db
> Reviewed-on: https://chromium-review.googlesource.com/815454
> Commit-Queue: Aaron Gable <agable@chromium.org>
> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
Bug: 792302
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Change-Id: I28d2260948aaf63bd865888c2f60e4cdee9aea48
Reviewed-on: https://chromium-review.googlesource.com/822990
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This reverts commit 9219d35688.
Reason for revert: unfortunately this says "core,quotePath" and since it includes recipe changes, we need something that the roller can munch on :(
Original change's description:
> Use core.quotePath=false when git is listing files
>
> This prevents git from putting quotes around some file names
> (those that have astral-plane characters) and not around others.
>
> R=maruel
>
> Bug: 792302
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Change-Id: I3b6a6b36c4720116de811b40177b59aa25c263db
> Reviewed-on: https://chromium-review.googlesource.com/815454
> Commit-Queue: Aaron Gable <agable@chromium.org>
> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
TBR=maruel@chromium.org,agable@chromium.org
Change-Id: I226388f19024403240a1443eb2b878b9293220e1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 792302
Reviewed-on: https://chromium-review.googlesource.com/821671
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
This prevents git from putting quotes around some file names
(those that have astral-plane characters) and not around others.
R=maruel
Bug: 792302
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Change-Id: I3b6a6b36c4720116de811b40177b59aa25c263db
Reviewed-on: https://chromium-review.googlesource.com/815454
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
$ git checkout <name>
may be interpreted by git as checking out a file named <name>
if <name> has slashes. Thus,
$ git checkout infra/config
ends up checking out infra/config directory of the (typically)
master branch that was checked out before. To avoid confusion,
we have to add '--' separator between branch name and file names:
$ git checkout <name> --
This in turns allows to run CQ-based presubmit on infra/config branches.
R=nodir@chromium.org
Bug: 790738
Change-Id: I6ce31a8f0fbd66fd59ac7c2ea9cccd3ff97d1f0e
Reviewed-on: https://chromium-review.googlesource.com/802136
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
bot_update and gclient invokes git cache with --ignore-locks.
This should also reduce noisiness of bot_update stdout.
R=iannucci@chromium.org
Bug:
Change-Id: I6baf9e5665d244d08aab58a7d3327e6c3204b2b5
Reviewed-on: https://chromium-review.googlesource.com/791536
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Tested end-to-end, for example
https://ci.chromium.org/swarming/task/3a0147207378b910
which contains:
src/media/cdm/api (Elapsed: 0:00:01)
----------------------------------------
[0:00:00] Started.
_____ src\media\cdm\api at ea5df8e78fbd0a4c24cc3a1f3faefefcd1b45237
[0:00:00] running "git cat-file -e ea5df8e78fbd0a4c24cc3a1f3faefefcd1b45237" in "e:\b\s\w\ir\cache\git\chromium.googlesource.com-chromium-cdm"
skipping mirror update, it has rev=ea5df8e78fbd0a4c24cc3a1f3faefefcd1b45237 already
thereby saving on needless git fetch (~40s in glcient sync on win trybots),
and reducing the rate of .pack file accumulation inside cache directories.
Risks: silently broken recipes which run gclient sync (or worse, bot_update)
as a means of fetching latest commits in all repos of a solution. I think
the benefit of faster bot_update in chromium CQ is worth the potential risk.
PSA: https://groups.google.com/a/chromium.org/d/msg/infra-dev/UYLdBwAXm1Y/OV9QB6JnBQAJ
Bug: 749709
Change-Id: I7a9e8ab82a5e2b848e450f19a798ac18a0b5e201
Reviewed-on: https://chromium-review.googlesource.com/787331
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Before, bot_update would print after command:
Succeeded (Failed) in 0.1 mins
but figuring out which command takes a lot of scrolling/searching
for previous
Running <cmd with args>
This CL fixes that:
Succeeded (Failed) in 0.1 mins <cmd with args>
R=machenbach@chromium.org, maruel@chromium.org
TEST=end-to-end with led tool, but only on LUCI stack.
Bug: 749709
Change-Id: Id3031a12b9013ab136ea03cdf6d325930722e50f
Reviewed-on: https://chromium-review.googlesource.com/787175
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
It used to be 50, I think ~9 gives best results for Chromium on Win:
on golo VM, it takes <4 minutes to re-boostrap + git fetch small
delta, assuming zipped git checkout for bootstrap is fresh (~1day).
For other repos, which are significantly smaller, this change should
have minor effect if at all.
Test: I tested this using `led` tool on Win7 machines running LUCI
stack extensively. For example,
* https://ci.chromium.org/swarming/task/3a0102e8c8657410
shows case with few .pack files, hence just 1 fetch
* https://ci.chromium.org/swarming/task/3a010282f9fd8010
shows case with 39 .pack files and so bootstrapping + fetch.
If you look at prior tasks on the same VM, you'd find this:
https://ci.chromium.org/swarming/task/39ffe843d01ed010
which spent 8 minutes doing 1 incremental fetch with 39 .pack
files.
**Troopers/Sheriffs**: This change is safe to revert.
However, beware that you should also at the same time revert the recipe
roll of this CL to the repo, in which the failed builder's recipe is
located, typically `chromium/tools/build`.
Bug: 749709
Change-Id: I18e2b63283100d466e9fb981a9094862463f6909
Reviewed-on: https://chromium-review.googlesource.com/787174
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Takuto Ikuta <tikuta@google.com>
This is a reland of 331a276b54
Original change's description:
> bot_update: make --output_json required in script invocation.
>
> It's already set by recipe, so this should have 0 difference in prod.
>
> R=iannucci@chromium.org
>
> Bug:
> Change-Id: I1e1129b9748d6647e45eb26892c8e892019fe088
> Reviewed-on: https://chromium-review.googlesource.com/769164
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Change-Id: I165da9650df6b5e8d382aafc30914eff9a524c56
Reviewed-on: https://chromium-review.googlesource.com/777820
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
This reverts commit 331a276b54.
Reason for revert: Looks like this is causing bots to fail on the waterfall
Example:
https://logs.chromium.org/v/?s=chromium%2Fbb%2Fchromium%2FWin%2F60913%2F%2B%2Frecipes%2Fsteps%2Fbot_update%2F0%2Fstdout
Original change's description:
> bot_update: make --output_json required in script invocation.
>
> It's already set by recipe, so this should have 0 difference in prod.
>
> R=iannucci@chromium.org
>
> Bug:
> Change-Id: I1e1129b9748d6647e45eb26892c8e892019fe088
> Reviewed-on: https://chromium-review.googlesource.com/769164
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
TBR=iannucci@chromium.org,tandrii@chromium.org
Change-Id: I65d7665e45dde539f6436ffe8932319e03f96f25
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/777619
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
It's already set by recipe, so this should have 0 difference in prod.
R=iannucci@chromium.org
Bug:
Change-Id: I1e1129b9748d6647e45eb26892c8e892019fe088
Reviewed-on: https://chromium-review.googlesource.com/769164
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Bug:
Change-Id: I031f5787a87a4bddbd3492912814476daf2c9b4f
Reviewed-on: https://chromium-review.googlesource.com/738118
Commit-Queue: William Hesse <whesse@google.com>
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
The test output was incorrect, this matches what it actually should look like.
Bug: 772529
Change-Id: I1ca9ceeb6d6b6f55b2ac7c90667c6a9904d2a43b
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: release_scripts
Change-Id: I1ca9ceeb6d6b6f55b2ac7c90667c6a9904d2a43b
Reviewed-on: https://chromium-review.googlesource.com/751694
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
This reverts commit 47b67c426b.
Reason for revert: Bug in bot_update.py resource, breaks recipe roller.
Original change's description:
> Reland "bot_update recipe: Upload source manifest"
>
> This reverts commit c3d1208d5c.
>
> Also:
> * Instead of replacing "manifest", just add a new "source_manifest"
> to the output JSON. This allow transition without breakage.
> * Change the test api so test for recipe output changes.
>
> The plan is to land this first, switch all downstream to "source_manifest",
> and then remove the original "manifest" key.
>
> Bug: 772529,776299
> Change-Id: Iffb75f18046f8e4c058afe077872d4257b9dd754
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Recipe-Nontrivial-Roll: skiabuildbot
> Recipe-Nontrivial-Roll: release_scripts
> Recipe-Nontrivial-Roll: skia
> Recipe-Nontrivial-Roll: skiabuildbot
> Recipe-Nontrivial-Roll: build
> Reviewed-on: https://chromium-review.googlesource.com/731378
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Ryan Tseng <hinoka@chromium.org>
TBR=iannucci@chromium.org,hinoka@chromium.org
Change-Id: I7a4ee904075e8b75b8a47f9ef0cd8a633af85a9c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 772529, 776299
Reviewed-on: https://chromium-review.googlesource.com/748312
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
This reverts commit c3d1208d5c.
Also:
* Instead of replacing "manifest", just add a new "source_manifest"
to the output JSON. This allow transition without breakage.
* Change the test api so test for recipe output changes.
The plan is to land this first, switch all downstream to "source_manifest",
and then remove the original "manifest" key.
Bug: 772529,776299
Change-Id: Iffb75f18046f8e4c058afe077872d4257b9dd754
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: skiabuildbot
Recipe-Nontrivial-Roll: release_scripts
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: skiabuildbot
Recipe-Nontrivial-Roll: build
Reviewed-on: https://chromium-review.googlesource.com/731378
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
If the revision has not yet propagated, then we never retry, because the amount of time to fetch HEAD already exceeds the deadline.
Bug: 771007
Change-Id: I91ac30b2b53b4f75729357b3298c52856a4d350e
Reviewed-on: https://chromium-review.googlesource.com/729328
Commit-Queue: Jao-ke Chin-Lee <jchinlee@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This fixes an issue introduced by
crrev.com/b40a45149a73a6b00e92fd1d2df4c0b1543d0005 where a 404 result
might return None contents.
R=agable@google.com, phajdan@google.com
Bug:774355
Change-Id: I993a029adb8d92cb27e043cd3d9b17cc8aee2432
Reviewed-on: https://chromium-review.googlesource.com/717937
Commit-Queue: Michael Moss <mmoss@chromium.org>
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
This is needed in the release recipes because there are times when we
want to check that a file is _not_ there yet (i.e. 404 response).
R=agable@chromium.org, dnj@chromium.org
Change-Id: Iedf646d6c73ebf898e90afd2009840b1e5c5b1ab
Reviewed-on: https://chromium-review.googlesource.com/709916
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Commit-Queue: Michael Moss <mmoss@chromium.org>
These aren't in use, and the original problem they were
meant to solve has been solved at the gclient.py layer
using resource locking:
https://codereview.chromium.org/2049583003
Bug: 773008
Change-Id: I6609f39d7f15604e0bb3d742a41c4f9fec87a57a
Reviewed-on: https://chromium-review.googlesource.com/707728
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
This is supported by gclient and has the effect of treating the solution
as a purely local, standalone DEPS (i.e. not as a source repo to
checkout or process in any way). It also prevents the solution from
being added to the output of 'gclient flatten'.
R=agable@chromium.org, phajdan.jr@chromium.org
Change-Id: Ie11246ec7a74097b96ea6a26983a63b3fb3123a5
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Reviewed-on: https://chromium-review.googlesource.com/698271
Commit-Queue: Michael Moss <mmoss@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
This is a step away from patch_project, which was a holdover
from Rietveld. Instead of mapping "projects" (which are
poorly-defined and not guaranteed to be unique) to subpaths,
instead map repository urls (which are at least unique).
Eventually we may be able to compute this directly from DEPS
instead of hardcoding the mapping here, but at least this is
a step in the right direction.
Bug: 765633
Change-Id: Idd65984fc6edefcbedb0438d38c2338b10b7e8e5
Reviewed-on: https://chromium-review.googlesource.com/690776
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
This arg used to be used in a 'sync' call that was removed in
crrev.com/7090f652d180eeff2663e3e3091d5db14123284c, but it was never
applied to the remaining 'sync' call. This removes the arg and leaves
"--with_branch_heads" (and adds "--with_tags") as default sync flags.
Change-Id: Ice5a592d95a0b09dafddab24769f5edd8835a1c9
Recipe-Manual-Change: build_limited_scripts_slave
Recipe-Manual-Change: build
Reviewed-on: https://chromium-review.googlesource.com/677754
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Commit-Queue: Michael Moss <mmoss@chromium.org>
Goal is to eventually add a presubmit trybot for skia_buildbot
repository.
Similar to one that was added in
https://codereview.chromium.org/1141583002
Bug: skia:7049
Change-Id: I95a8e2b452a44a50dc177d520af4ce95129db6da
Reviewed-on: https://chromium-review.googlesource.com/671410
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Ravi Mistry <rmistry@chromium.org>
Bug: 765633
Change-Id: I8f2a9ba807fbe176695a7d44467f3948eea07fda
Reviewed-on: https://chromium-review.googlesource.com/668446
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Henrik Kjellander <kjellander@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Print the Python version during "bot_update". This is useful for
visually auditing the environment in which things are running.
BUG=None
TEST=None
R=iannucci@chromium.org
Change-Id: I92be8a888e6f1e5c17205665dff36378e641cd81
Reviewed-on: https://chromium-review.googlesource.com/636450
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
R=agable@chromium.org, dnj@chromium.org, vadimsh@chromium.org
Bug: 726507
Change-Id: I8cc5619a5fe93c76ffff32a8915c8b9bf3e077d3
Reviewed-on: https://chromium-review.googlesource.com/607649
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
The only user of this method needs to apply two patches from two
different repos in a try job. Extending the method to handle this.
BUG=628259
R=agable@chromium.org
Change-Id: Ia119780d1323add0094631e5c291b0e7a573e6e5
Reviewed-on: https://chromium-review.googlesource.com/588008
Commit-Queue: Sergey Berezin <sergeyberezin@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
The query parameters are parsed out of a string using
urlparse.parse_qs, which produces a dictionary whose values
are lists of values.
The query parameters are then stuffed back into a string
using urlencode.urlencode, which uses percent-encoding to
encode lists as literal sequences of brackets and commas,
resulting in parse_qs and urlencode not being inverse
operations.
This change adds the doseq flag to urlencode, thus making
it encode each value from the list of values as a separate
key=value pair, thus making it a true inverse of parse_qs.
R=dnj@chromium.org
Change-Id: Ic4631cc672f9e42ba371306efdda71892d625119
Reviewed-on: https://chromium-review.googlesource.com/575398
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
This was removed in downstream recipes after it was turned on by default.
Bug: 740456
Change-Id: Idae1905af97333970c8b31508634f8b71550c6f9
Reviewed-on: https://chromium-review.googlesource.com/574030
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
This adds angle to the got_revision_mapping to take account of the checked
out revision and use it correctly in the without-patch application.
This just adds the got_revision property to all test cases in the output
and fixes the second bot_update run of two angle test cases.
Bug: 740456
Change-Id: I6ae6bf627653280bc2a94230ec77949411dc4331
Reviewed-on: https://chromium-review.googlesource.com/573741
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Update the CIPD recipe module to generate platform suffixes for
arbitrary OS, architecture, and bitness.
BUG=None
TEST=expectations
Change-Id: Ic713adc403dc135895a1cfaaa6ba1546742125d9
Reviewed-on: https://chromium-review.googlesource.com/573412
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
The previously passed branch:HEAD notation is understood by bot_update
for the main project, but not by gclient for deps'ed projects.
We don't need the colon notation at all as passing a branch implies
using the HEAD of that branch.
Bug: 740456
Change-Id: I95eb88f0de2e06bee8a3e7db24c4ad85cdb76d3e
Reviewed-on: https://chromium-review.googlesource.com/566866
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Several tools, including the "git" recipe module, hard-code a
checkout-relative "git.bat" path. Git is a feature that is provided by
the system, both to tooling and recipes:
1) For users, "depot_tools" must be on PATH, and during setup it will
have installed "git.bat", ensuring that Git tooling is available in
PATH.
2) For bots, the system is responsible for providing "git.bat" on PATH.
This is typically done at "/b/depot_tools/git.bat", which is sync'd
through the "update_scripts" step.
By formally treating Git as a system resource, we absolve Windows bots
and users from manually installing a depot_tools-local Git, bringing
them in line with other platforms.
BUG=chromium:590806
TEST=local
Change-Id: I93e89855cdd330a2ba7a8cfb8117a1789d1ab54e
Reviewed-on: https://chromium-review.googlesource.com/568694
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
This makes bot_update query Gerrit when syncing for Gerrit tryjobs.
The query will establish the actual destination branch of the CL,
which can be different from master (e.g. a feature branch).
Bot_update will ensure to use this destination branch for the repo
that corresponds to the CL's project. Both the main project or a
deps'ed project work.
Initially, this lives behind a flag that can be controlled in
downstream recipes. Eventually we'll set this to default after a
gradual roll-out.
Branches in branch-heads are not supported yet.
Bug: 740456
Change-Id: I4a0d50e2ca8fe90f8d29964a3ffab17291f7be60
Reviewed-on: https://chromium-review.googlesource.com/566824
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
The /a suffix in the hots URL is not necessary as the gerrit_util adds it by
default to authenticate. Having it hard-coded in the example can be misleading.
Bug: 685318
Change-Id: I333cd8b2aa9020aadfd186f2e18fbff0aa917681
Reviewed-on: https://chromium-review.googlesource.com/564611
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
This'll allow to query gerrit from recipes for e.g. current changes in
CQ.
Bug: 685318
Change-Id: I73d08d4b186b2e5fe044fd4d4fafd9db62e27066
Reviewed-on: https://chromium-review.googlesource.com/558939
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
"bot_update" currently passes a full "gclient" specification as a
command-line argument to "bot_update.py". This file contains all sorts
of junk that clutters the command-line, and revealed a batch file
escaping bug that seems to be present in the recipe engine. Change this
so that "bot_update" passes the spec as file content instead of a
command-line flag.
Also, "bot_update" apparently called "--spec", which optparse kindly
recognized as the real flag, "--specs". This has always(?) been a bug,
and is no longer a problem.
BUG=chromium:738348
TEST=expectations
Change-Id: I5324b94dd5ef78503e7cfdc28b5fde8f434e4359
Reviewed-on: https://chromium-review.googlesource.com/558207
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Prefix "depot_tools" to PATH instead of appending. This will ensure that
"depot_tools" pulls resources from the recipe checkout as a preference,
not as a last resort.
Use "env_prefixes" instead of "env" to prefix PATH.
Remove Git speed limit settings, which are taken care of by the Git
wrapper now.
BUG=chromium:738176
TEST=expectations
R=hinoka@chromium.org, iannucci@chromium.org
Change-Id: I2e509a331266e867be323bc70e3bc0f08f85246f
Reviewed-on: https://chromium-review.googlesource.com/556287
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
"remote_run" builds explicitly supply a cleanup directory in recipe
properties, so it isn't necessary to supply it here.
BUG=chromium:725631
TEST=None
TBR=iannucci@chromium.org
Change-Id: I6eaaeea14bd699181d7ceeff6b3baac2d1cd9861
Reviewed-on: https://chromium-review.googlesource.com/528630
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Currently, "bot_update" relies on a BuildBot cleanup mechanism and, to a
lesser extent, the standard BuildBot directory layout. Both of these are
problematic when projecting it into other circumstances, notably
"remote_run" and LUCI.
Have "bot_update" handle its own cleanup. It will now choose a cleanup
directory within the hierarchy of its checkout, and explicitly purge it
prior to execution if it exists. This enforces its expected behavior in
all circumstances and removes its expectations of the greater checkout
layout.
Export "cleanup_dir" via "infra_paths" to point to "build.dead" when
running on BuildBot builds. Otherwise, it is a default directory which,
on Kitchen, is ephemeral by design.
BUG=chromium:725631
TEST=expectations
Change-Id: I664434c542a25aaa7ff3eac216208a2425730fde
Reviewed-on: https://chromium-review.googlesource.com/528057
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).
Please review the expectation changes, and LGTM+CQ.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
https://crrev.com/154ef7e5d8f964ef30b1250dd33751582705f6f2 [path] Add concept of a "cleanup" directory. (dnj@chromium.org)
R=dnj@chromium.org, martiniss@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I31787615cfaa6f33deae6c1c0c3d7b748c222d05
Reviewed-on: https://chromium-review.googlesource.com/527503
Commit-Queue: Recipe Roller <recipe-roller@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Daniel Jacques <dnj@chromium.org>
`parsed_root.path` already contains the leading slash in the the URL, so adding
an additional slash gives an incorrect URL.
Change-Id: I84f04179356bb132a31531232a5c30c7808f7c1b
Reviewed-on: https://chromium-review.googlesource.com/520724
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: David Tu <dtu@chromium.org>
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).
Please review the expectation changes, and LGTM+CQ.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
https://crrev.com/458ce956c9d91d184e0107c5fa6b34a8267b8e9b [step_runner] run _merge_envs in simulation too. (iannucci@chromium.org)
R=martiniss@chromium.org, phajdan.jr@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I0d60499b06718827776739eb0413a79baa15cd9e
Reviewed-on: https://chromium-review.googlesource.com/520462
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Some recipes interacting with older revisions (e.g. bisect)
will need this.
Bug: 570091
Change-Id: I38e5ffa2db1a9bfae3667f015f00977c32ebe51e
Reviewed-on: https://chromium-review.googlesource.com/519407
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Currently, the "cipd" recipe module allows a package to be created by
adding directories to a PackageDefinition via "add_dir". However, there
is no current way to add the entirety of a root directory to a package.
The PackageDefinition will generate a relative path of "''", which CIPD
will reject as empty.
Make it so that if the user supplies the root directory as the directory
to add to the package, it properly includes it as ".".
BUG=None
TEST=local
- Ran recipe with this modification, seems to work
R=iannucci@chromium.org, nodir@chromium.org, vadimsh@chromium.org
Change-Id: Ib5c72038cf153776808f084db835c60f61fd9044
Reviewed-on: https://chromium-review.googlesource.com/510309
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
All known repositories have been switched to "recipe_engine/url".
BUG=None
TEST=None
R=iannucci@chromium.org
Change-Id: Ibed376b916f59b5d7142aa2ffd210919079a0862
Reviewed-on: https://chromium-review.googlesource.com/508163
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Retries are no longer necessary, since the Git wrapper will retry all
transient Git operations.
BUG=None
TEST=None
R=hinoka@chromium.org, iannucci@chromium.org
Change-Id: I18f066004992c98e54665833360944cc7d6550cc
Reviewed-on: https://chromium-review.googlesource.com/506375
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
The Git wrapper now implicitly retries, so this call is unnecessary.
BUG=chromium:721450
TEST=None
R=hinoka@chromium.org, iannucci@chromium.org
Change-Id: I79e42aa050f6fec14506b685d379a76e8296fea3
Reviewed-on: https://chromium-review.googlesource.com/506493
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
This removes remaining configs that are tied to the
chromium recipe_module.
This directly depends on:
https://chromium-review.googlesource.com/c/497449/
In case of a revert, both CLs should be reverted.
Bug: 717457
Change-Id: I35ef5c480a08d54c27f7545f94ebee3b8d20374f
Reviewed-on: https://chromium-review.googlesource.com/497432
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
This directly depends on adding these configs to build:
https://chromium-review.googlesource.com/c/496187/
The CL adding the configs to build must also roll depot_tools to this CL.
In case of a revert, both CLs must be reverted.
Bug: 717457
Change-Id: I458b52e184b668a3ace18ea0948363a2795f7161
Reviewed-on: https://chromium-review.googlesource.com/496107
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
This migrates chromium to the new mapping after testing it for V8 in
https://chromium-review.googlesource.com/485579
Also switches dependent gclient configs. This will be a noop downstream.
Bug: 713356
Change-Id: Ic1cb4de139895021dfd35063a9d6c8e7a4e77b9d
Reviewed-on: https://chromium-review.googlesource.com/488604
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This reverts commit c91686c087.
Reason for revert: Reland a light version of the original CL that keeps a dummy flag around.
Original change's description:
> Revert "Remove obsolete bot_update flag"
>
> This reverts commit 72048266d5.
>
> Reason for revert: Skia depends on it. Need to clean that up first. Currently blocks roll:
> https://skia-review.googlesource.com/c/14145/
>
> Original change's description:
> > Remove obsolete bot_update flag
> >
> > Using the manifest is the default since a long time. The flag was only introduced to smoothly switch it on. Now the flag is not needed anymore.
> >
> > Bug: 398105
> > Change-Id: I4eb1c98e26e025a69f8b0a7c48877d78c031702e
> > Reviewed-on: https://chromium-review.googlesource.com/485659
> > Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> > Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
> > Commit-Queue: Michael Achenbach <machenbach@chromium.org>
> >
>
> TBR=phajdan.jr@chromium.org,machenbach@chromium.org,tandrii@chromium.org,sergiyb@chromium.org,chromium-reviews@chromium.org
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
>
> Change-Id: Iffc4d8076abd3c92feebfd7f3ef88be7ce34b692
> Reviewed-on: https://chromium-review.googlesource.com/486639
> Reviewed-by: Michael Achenbach <machenbach@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
>
R=phajdan.jr@chromium.org,machenbach@chromium.org,tandrii@chromium.org,sergiyb@chromium.org,chromium-reviews@chromium.org
Bug: 398105
Change-Id: I817d446f8a8eff84c5d7143ac2b9c527a40991f1
Reviewed-on: https://chromium-review.googlesource.com/486640
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).
Please review the expectation changes, and LGTM+CQ.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
https://crrev.com/944125e6d1e8c831d09517bde658a38d8f81db37 Sort step presentation properties (machenbach@chromium.org)
R=martiniss@chromium.org, phajdan.jr@chromium.org
Change-Id: Iec279f8331bd0e7673ce785c3095714bc5cfe401
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Reviewed-on: https://chromium-review.googlesource.com/486600
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
This reverts commit 72048266d5.
Reason for revert: Skia depends on it. Need to clean that up first. Currently blocks roll:
https://skia-review.googlesource.com/c/14145/
Original change's description:
> Remove obsolete bot_update flag
>
> Using the manifest is the default since a long time. The flag was only introduced to smoothly switch it on. Now the flag is not needed anymore.
>
> Bug: 398105
> Change-Id: I4eb1c98e26e025a69f8b0a7c48877d78c031702e
> Reviewed-on: https://chromium-review.googlesource.com/485659
> Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
> Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
> Commit-Queue: Michael Achenbach <machenbach@chromium.org>
>
TBR=phajdan.jr@chromium.org,machenbach@chromium.org,tandrii@chromium.org,sergiyb@chromium.org,chromium-reviews@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
Change-Id: Iffc4d8076abd3c92feebfd7f3ef88be7ce34b692
Reviewed-on: https://chromium-review.googlesource.com/486639
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Using the manifest is the default since a long time. The flag was only introduced to smoothly switch it on. Now the flag is not needed anymore.
Bug: 398105
Change-Id: I4eb1c98e26e025a69f8b0a7c48877d78c031702e
Reviewed-on: https://chromium-review.googlesource.com/485659
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Fix the bug in cipd recipe module preventing to add the package root as a dir.
R=vadimsh@chromium.org
Bug:
Change-Id: Ia9b7adbf7f9b0e0e6dd4158934197e4f69ba993e
Reviewed-on: https://chromium-review.googlesource.com/483850
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
A few rare recipes use two or more solutions side-by-side.
This allows switching one solution in isolation to use the new mapping.
Bug: 713356
Change-Id: Id51f2f777681401843004577a4de65a6c5d1e4a6
Reviewed-on: https://chromium-review.googlesource.com/484341
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This is the first step in reversing got_revision_mapping.
This CL:
- Adds a new configuration got_revision_reverse_mapping.
- Ensures only the old or the new configuration is used.
- Transforms the old to the new configuration within bot_update.
In follow up CLs we will migrate all clients to the new mapping, then delete the old and rename the new again.
Bug: 713356
Change-Id: Icdbd6a3748c6e5399b9c4867a3d8292973325160
Reviewed-on: https://chromium-review.googlesource.com/483479
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Add "tools/build"'s "url" and "gitiles" recipe modules into
"depot_tools". "gitiles" had an explicit dependency on "depot_tools", so
this formalizes this dependency.
To support "url", we vendor "requests" from "tools/build" into
"depot_tools" for it.
BUG=None
TEST=expectations
Change-Id: I1ba634ae264b5c9d3adee20db3397c83a6c52c45
Reviewed-on: https://chromium-review.googlesource.com/483316
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
This reverts commit e8acd40a11.
Reason for revert: broke windows builds :(
Original change's description:
> bot_update: pin depot_tools
>
> gclient runhooks often calls scripts that depend on scripts in depot_tools.
> Currently bot_update only ensures that depot_tools is present in $PATH, but
> otherwise uses whatever depot_tools already specified in $PATH. We don't pin
> depot_tools before running a recipe, so using whatever depot_tools in $PATH
> is not hermetic.
>
> Insert the path to the pinned depot_tools to the beginning of $PATH
>
> R=iannucci@chromium.org
>
> Bug:
>
> Change-Id: I31d04ba2ec7c409f89d58a51502ce3565e3a02a1
> Reviewed-on: https://chromium-review.googlesource.com/481072
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Nodir Turakulov <nodir@chromium.org>
>
TBR=iannucci@chromium.org,nodir@chromium.org,phajdan.jr@chromium.org,chromium-reviews@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
Change-Id: I1a70f0f54e9b9cea7fc2f9897e62f558204a4684
Reviewed-on: https://chromium-review.googlesource.com/481440
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
gclient runhooks often calls scripts that depend on scripts in depot_tools.
Currently bot_update only ensures that depot_tools is present in $PATH, but
otherwise uses whatever depot_tools already specified in $PATH. We don't pin
depot_tools before running a recipe, so using whatever depot_tools in $PATH
is not hermetic.
Insert the path to the pinned depot_tools to the beginning of $PATH
R=iannucci@chromium.org
Bug:
Change-Id: I31d04ba2ec7c409f89d58a51502ce3565e3a02a1
Reviewed-on: https://chromium-review.googlesource.com/481072
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Have "bot_update.py" export the failing patch body in its JSON. Have the
recipe actually emit the patch body in the "Patch Failure" step instead
of the "bot_update" step, and remove the "Patch Failure" step's
indirection to the "bot_update" step for patch failure.
This will provide a more straightforward user interface: the red step
that says "Patch Failure" now includes the details of the patch failure!
BUG=chromium:710477
TEST=expectations
Change-Id: I6ad305f0b972de89e4f0b5eb70edf41980447abd
Reviewed-on: https://chromium-review.googlesource.com/475950
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Bug: 710817
Change-Id: Ifb2a9abe605d9e403c55ec1360d186af12dd2f34
Reviewed-on: https://chromium-review.googlesource.com/475775
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Bug: 710817
Change-Id: Ice7cefaf5c5a5cac431a95d3adfa44f323eafb2f
Reviewed-on: https://chromium-review.googlesource.com/475230
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
It proved to be dangerous to introduce conditional logic into recipes
and recipe modules to alternate between buildbot and luci modes, namely
it touches buildbot code paths, which can be very dangerous.
Instead, introduce "generic" path config that configures
existing base paths from the built-in ones.
Also remove swarmbucket path config because it is not used.
--
This CL is a simpler version of
https://chromium-review.googlesource.com/c/459234/
in that it does not make generic path config a default.
This avoids behavioral change in downstream recipe packages and
makes rolling into build.git trivial.
R=iannucci@chromium.org
BUG=660481
Change-Id: Ifc147e29e68a6e0c800da73a8e356238dfc2938a
Reviewed-on: https://chromium-review.googlesource.com/461359
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
This reverts commit 8b5b594115.
Reason for revert: nothing wrong with this CL, but implementing a manual roll to build.git takes more than 20 min. I will reland this CL once my roll to build.git CL is ready
Original change's description:
> add "generic" infra path config
>
> It proved to be dangerous to introduce conditional logic into recipes
> and recipe modules to alternate between buildbot and luci modes, namely
> it touches buildbot code paths, which can be very dangerous.
>
> Instead, introduce "generic" default path config that configures
> existing base paths from the built-in ones.
>
> Also remove swarmbucket path config because it is not used.
>
> R=iannucci@chromium.org
> BUG=660481
>
> Change-Id: Ide0e1f64913fe35766b997ab5b55f01f4c63aa58
> Reviewed-on: https://chromium-review.googlesource.com/459234
> Commit-Queue: Nodir Turakulov <nodir@chromium.org>
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
>
TBR=iannucci@chromium.org,nodir@chromium.org,chromium-reviews@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=660481
Change-Id: If27a537689dfe3669015bcbc7a5f07ca1f5c10af
Reviewed-on: https://chromium-review.googlesource.com/461223
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
It proved to be dangerous to introduce conditional logic into recipes
and recipe modules to alternate between buildbot and luci modes, namely
it touches buildbot code paths, which can be very dangerous.
Instead, introduce "generic" default path config that configures
existing base paths from the built-in ones.
Also remove swarmbucket path config because it is not used.
R=iannucci@chromium.org
BUG=660481
Change-Id: Ide0e1f64913fe35766b997ab5b55f01f4c63aa58
Reviewed-on: https://chromium-review.googlesource.com/459234
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
This will help remove confusion when depot_tools' recipes.py
ends up in PATH.
BUG=699120
Change-Id: Id4c21b0cc6bb022ea2c21145abe76bebb0a8d9c1
Reviewed-on: https://chromium-review.googlesource.com/458430
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Yoshisato Yanagisawa
Stephen Martinis
Robert Iannucci
Scott Graham
John Budorick
Ryan Tseng
Edward Lemur
Nodir Turakulov
Nodir Turakulov
Corentin Wallez
Ben Pastene
Andrii Shyshkalov
recipe-roller
Michael Achenbach
Sergiy Byelozyorov
Andrew Grieve
Takuto Ikuta
Stephen Martinis
Michael Moss
Nico Weber
Tom Anderson
Oleh Prypin
Lei Zhang
Takuto Ikuta
Aaron Gable
Robert Flack
William Hesse
Ryan Tseng
Jao-ke Chin-Lee
Vadim Shtayura
Emma
Ravi Mistry
Dan Jacques
Robert Iannucci
Sergey Berezin
Daniel Jacques
Paweł Hajdan, Jr
Dave Tu