When syncing remote branches, checkout origin/<branch> instead of
<branch>, since there might be a local branch with that name.
Bug: 859703
Change-Id: I9a49b940f53f75a97e5458fdcf578c3b52b26de2
Reviewed-on: https://chromium-review.googlesource.com/1128271
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Today, whenever LUCI scheduler sets gitiles_commit, a revision property
is also always set, thus this CL will have no effect on today's production.
The goal of CL is to pave the way to avoid this duplication. For starters,
one could start real writing tests with
+ api.buildbucket.ci_build(..., revision='sha1')
without the need to duplicate the same revision as a property. All
recipes which WAI with such input are thus ready to work w/o 'revision'
property. After that, we'll be able to stop the duplication in LUCI scheduler.
R=hinoka
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Change-Id: I1773b1c3d2a98a9dbd1cfdd9ac2b4fb99745ee99
Reviewed-on: https://chromium-review.googlesource.com/1123713
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
gclient should be the one syncing the solutions to the appropriate
revisions.
Bug: 850812, 853032
Change-Id: Ieefc5661627d4864deb0d4e7053168a99da29d29
Reviewed-on: https://chromium-review.googlesource.com/1102833
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
shallow mode is a special git mode that only checks out a shallow copy
of git so the full history is not downloaded. It is not well supported
by googlesource.com servers because git has to do a giant negotiation
with the server and transmit every single commit that the local checkout
has or doesn't have (instead of just the latest commit of each ref),
which ends up taking more than 10x longer than a non-shallow checkout.
R=ehmaldonado, hinoka, iannucci
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Bug: 855137
Change-Id: I5f9e31f8b2730cf80b79bfd0e08201c33eb942f5
Reviewed-on: https://chromium-review.googlesource.com/1110450
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
It fails when both the revision and the url are None.
Bug: 841936
Change-Id: Idef45a015624a92226d4ccd38ed5b978bf786993
Reviewed-on: https://chromium-review.googlesource.com/1053996
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
This is a reland of 3806b7fbd0
Original change's description:
> bot_update: Fetch references to be synced.
>
> If a solution is to be synced to a reference, such as
> refs/branch-heads/#### fetch it so that it can later be
> checked out.
>
> I tried this change on a LUCI task that runs chromium_presubmit
> for crrev.com/c/1048868. The bot_update works as expected:
> https://ci.chromium.org/swarming/task/3d5b294cff467710?server=chromium-swarm.appspot.com
>
>
> Bug: 840445
>
> Change-Id: I6444c0feb679ca89b2f4065215de915494e5082a
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Reviewed-on: https://chromium-review.googlesource.com/1049007
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Bug: 840445
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Change-Id: I64a97aafe5fab9a26c478262e9ca104dc59e61f5
Reviewed-on: https://chromium-review.googlesource.com/1052387
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
This reverts commit 3806b7fbd0.
Reason for revert: Broke downstream tot bots
Original change's description:
> bot_update: Fetch references to be synced.
>
> If a solution is to be synced to a reference, such as
> refs/branch-heads/#### fetch it so that it can later be
> checked out.
>
> I tried this change on a LUCI task that runs chromium_presubmit
> for crrev.com/c/1048868. The bot_update works as expected:
> https://ci.chromium.org/swarming/task/3d5b294cff467710?server=chromium-swarm.appspot.com
>
>
> Bug: 840445
>
> Change-Id: I6444c0feb679ca89b2f4065215de915494e5082a
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Reviewed-on: https://chromium-review.googlesource.com/1049007
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
TBR=mmoss@chromium.org,machenbach@chromium.org,tandrii@chromium.org,ehmaldonado@chromium.org
Change-Id: If94df8684e634504d8a0c11b2aa5862a458ba19f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 840445, 841306
Reviewed-on: https://chromium-review.googlesource.com/1052187
Reviewed-by: agrieve <agrieve@chromium.org>
Commit-Queue: agrieve <agrieve@chromium.org>
If a solution is to be synced to a reference, such as
refs/branch-heads/#### fetch it so that it can later be
checked out.
I tried this change on a LUCI task that runs chromium_presubmit
for crrev.com/c/1048868. The bot_update works as expected:
https://ci.chromium.org/swarming/task/3d5b294cff467710?server=chromium-swarm.appspot.com
Bug: 840445
Change-Id: I6444c0feb679ca89b2f4065215de915494e5082a
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Reviewed-on: https://chromium-review.googlesource.com/1049007
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This is a reland of e86fe982e8
Original change's description:
> bot_update: Set user.{name,email} when applying patch refs.
>
> git rebase complains when user name/email has not been configured, as was
> the case for the Chromium trybot.
>
> Bug: 643346
> Change-Id: Iab7dac0a98608e09b63a53d5bacb98925701fbfd
> Reviewed-on: https://chromium-review.googlesource.com/1031524
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Bug: 643346
Change-Id: I4b4d07675222d39d704560ed84f9c221e87aba8b
Reviewed-on: https://chromium-review.googlesource.com/1036309
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This is needed for release builders to checkout the release tags that
will be created when the buildspec branch/release info is migrated out
to the regular repos.
BUG=825063
R=jbudorick@google.com
Change-Id: I5f0e4cdd94de11c59f9b4a5c11aabea129e81523
Reviewed-on: https://chromium-review.googlesource.com/1036526
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Michael Moss <mmoss@chromium.org>
This reverts commit e86fe982e8.
Reason for revert: Probably causing outages on win7_chromium_rel_ng.
Original change's description:
> bot_update: Set user.{name,email} when applying patch refs.
>
> git rebase complains when user name/email has not been configured, as was
> the case for the Chromium trybot.
>
> Bug: 643346
> Change-Id: Iab7dac0a98608e09b63a53d5bacb98925701fbfd
> Reviewed-on: https://chromium-review.googlesource.com/1031524
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
TBR=agable@chromium.org,ehmaldonado@chromium.org
Change-Id: Ic57398323476fccef34f766270d760c3975630a6
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 643346
Reviewed-on: https://chromium-review.googlesource.com/1036308
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
git rebase complains when user name/email has not been configured, as was
the case for the Chromium trybot.
Bug: 643346
Change-Id: Iab7dac0a98608e09b63a53d5bacb98925701fbfd
Reviewed-on: https://chromium-review.googlesource.com/1031524
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Previously, anything not equal to HEAD meant pinned revision.
This backfired on values like "origin/master" which are meant as
equivalent to HEAD, but which bot_update tried to ensure exist in
checkout, which was refreshed several times under assumption that
this is eventual consistency of git servers.
Instead, this CL treats repo as pinned if and only if the treeish
of the repo is matching sha1 regex.
R=hinoka@chromium.org
Bug: 838292
Change-Id: I9cb46535f46b2898cf54900ede7960ba08770d77
Post-Scriptum: git bisect + Led is awesome, took only 8 manual steps.
Reviewed-on: https://chromium-review.googlesource.com/1035793
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
This reverts commit 355b11a31f.
Reason for revert:
Easier to do this another way that doesn't cause expectation changes,
so it doesn't require non-trivial rolls.
Original change's description:
> bot_update: Patch on gclient by default.
>
> See [4] for the design doc for this change.
>
> We're moving the patching logic from bot_update to gclient.
> This way we can test patches of dependencies that modify their DEPS
> files as well, for example when ANGLE uploads a patch that modifies
> DEPS files to be tested by a Chromium trybot.
>
> Chromium Linux CQ and Angle trybots are already running this way
> (see [1] and [2]) and a PSA has already been sent [3]
>
> [1] https://chromium.googlesource.com/chromium/src/+/master/infra/config/global/cr-buildbucket.cfg#214
> [2] https://chromium.googlesource.com/angle/angle/+/infra/config/cr-buildbucket.cfg#105
> [3] https://groups.google.com/a/google.com/forum/#!topic/chrome-infra/FVHN2kdUX4I
> [4] https://docs.google.com/document/d/1GSTw4ysm5AYjNR6vMA7NJUxLK2VN9IpOEp_v0u-wyOE/edit?ts=5ad78165#
>
> Bug: 643346
> Change-Id: Ic9a9dcfae86aca730d35fbb26c26b21135e05bf3
> Recipe-Nontrivial-Roll: skia
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Recipe-Nontrivial-Roll: skiabuildbot
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: release_scripts
> Reviewed-on: https://chromium-review.googlesource.com/1014356
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>
TBR=agable@chromium.org,ehmaldonado@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 643346
Change-Id: I0e694b0e398770b8bd2e5a3170e158a7e54c5a61
Reviewed-on: https://chromium-review.googlesource.com/1029310
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
We should apply the patch only when gerrit_repo and gerrit_ref are passed,
instead of using only the --apply-patch-on-gclient flag.
R=agable@chromium.org
Bug: 643346
Change-Id: I28547aed9daeefa2657a108e0bacbfd4fc9fa07e
Reviewed-on: https://chromium-review.googlesource.com/989667
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This adds a --enable-gclient-experiment flags that tells bot_update.py
to skip applying the patch, and instead forward the flags to gclient.
Bug: 643346
Change-Id: Ia4275a126e6adba54dfcc894d224c50c166db90e
Reviewed-on: https://chromium-review.googlesource.com/962938
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
This method hasn't been supported for a long time now (and besides, its
only for Rietveld).
R=agable@chromium.org, tandrii@chromium.org
Recipe-Manual-Change: infra
Change-Id: Ie6e63834dca67962db29f2cb407950ed85db55a7
Reviewed-on: https://chromium-review.googlesource.com/957832
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Also, some lines went over 80chars, so I ran 'yapf --style chromium' which
formatted some unrelated things.
BUG=905285
R=dpranke
Change-Id: Iee5f46d88a6e9782612cc4f9e5a2cb72d62ab6af
Reviewed-on: https://chromium-review.googlesource.com/907736
Commit-Queue: Thomas Anderson <thomasanderson@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
This reverts commit fa573785df.
Reason for revert: broke CQ on branches.
Example: https://ci.chromium.org/p/chromium/builders/luci.chromium.try/chromium_presubmit/28220
Original change's description:
> bot_update: allow rebasing the patch onto an older revision.
>
> When applying a rebase, we normally go from an older base commit to a newer one.
>
> A ---- B ---- C ---- D -origin/master
> \
> E -branch
>
> In this case, `git rebase D` would certainly work as expected. However,
> writing `git rebase B` would NOT get us to the following state:
>
> A ---- B ---- C ---- D -origin/master
> \
> E' -branch
>
> In fact, it would have no effect.
>
> This article http://matthew-brett.github.io/pydagogue/rebase_without_tears.html
> explains the general invocation as
> > `git rebase --onto <graft-point> <exclude-from> <include-from>`
> > If you don’t specify --onto, <graft-point> defaults to <exclude-from>
>
> So what's happening is, by writing `git rebase B` we're rebasing onto B,
> excluding commits that are in B. Commit C is not "in" B so it is kept and we're
> back to the starting point.
>
> So I suggest to change the invocation to `git rebase --onto B origin/master`,
> which rebases onto B, excluding commits that are in origin/master. This works
> more generally and allows rebasing "backwards".
>
> Bug: None
> Change-Id: I68e4d805811530b585550bc75099354fef4e9c15
> Reviewed-on: https://chromium-review.googlesource.com/904004
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
> Commit-Queue: Oleh Prypin <oprypin@chromium.org>
TBR=iannucci@chromium.org,tandrii@chromium.org,oprypin@chromium.org
Change-Id: I7243641d84428bf4504ee798bf31ad0afbfd4865
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: None
Reviewed-on: https://chromium-review.googlesource.com/905862
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Goal: reduce number of git fetches from remote to the minimum possible.
Minimum possible is either:
* 0 if desired revision is pinned (!= HEAD) and already exists in cache
* 1 if desired revision is HEAD
* >= 2 if revision is pinned but we talk temporarily
out-of-date git server not yet having it.
This CL achieves the above.
No fetch:
https://ci.chromium.org/swarming/task/39dad20230c84110?server=chromium-swarm.appspot.com
bot_update takes <1min, but it used to take at least 2. The steps are now like this:
...
===Running git cat-file -e 33bf8a94dcd5e0abbdf83e4afaed24b5180e3eb2===
In directory: /b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src
===Succeeded in 0.0 mins===
===Running /b/swarming/w/ir/cipd_bin_packages/bin/python -u /b/swarming/w/ir/recipe-checkout-dir/depot_tools/git_cache.py exists --quiet --cache-dir /b/swarming/w/ir/cache/git https://chromium.googlesource.com/chromium/src.git===
In directory: /b/swarming/w/ir/cache/builder/linux
/b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src
===Succeeded in 0.0 mins===
===Running git remote set-url origin /b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src===
In directory: /b/swarming/w/ir/cache/builder/linux/src
===Succeeded in 0.0 mins===
===Running git fetch origin===
In directory: /b/swarming/w/ir/cache/builder/linux/src
From /b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src
003eedbfee47..5210107a20e8 master -> origin/master
===Succeeded in 0.0 mins===
===Running git checkout --force 33bf8a94dcd5e0abbdf83e4afaed24b5180e3eb2===
...
Fetch & success:
https://ci.chromium.org/swarming/task/39db022b874ffd10?server=chromium-swarm.appspot.com
Fetch & retry because missing rev:
https://ci.chromium.org/swarming/task/39db04ce8b4b8910?server=chromium-swarm.appspot.com
Fetch because rev=HEAD:
https://ci.chromium.org/swarming/task/39db0c0dd5ab5d10?server=chromium-swarm.appspot.com
This has already been adopted in gclient.
Bug:
Change-Id: Id99892b62719fdf3f7e6e59058986d1500384f8d
Reviewed-on: https://chromium-review.googlesource.com/771591
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
When applying a rebase, we normally go from an older base commit to a newer one.
A ---- B ---- C ---- D -origin/master
\
E -branch
In this case, `git rebase D` would certainly work as expected. However,
writing `git rebase B` would NOT get us to the following state:
A ---- B ---- C ---- D -origin/master
\
E' -branch
In fact, it would have no effect.
This article http://matthew-brett.github.io/pydagogue/rebase_without_tears.html
explains the general invocation as
> `git rebase --onto <graft-point> <exclude-from> <include-from>`
> If you don’t specify --onto, <graft-point> defaults to <exclude-from>
So what's happening is, by writing `git rebase B` we're rebasing onto B,
excluding commits that are in B. Commit C is not "in" B so it is kept and we're
back to the starting point.
So I suggest to change the invocation to `git rebase --onto B origin/master`,
which rebases onto B, excluding commits that are in origin/master. This works
more generally and allows rebasing "backwards".
Bug: None
Change-Id: I68e4d805811530b585550bc75099354fef4e9c15
Reviewed-on: https://chromium-review.googlesource.com/904004
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Oleh Prypin <oprypin@chromium.org>
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).
Please review the expectation changes, and LGTM+CQ.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
https://crrev.com/77667b5ba38c4d205a743d04eb625ac068bdd78e Always use Gerrit for tryserver recipe tests (agable@chromium.org)
R=dnj@chromium.org, martiniss@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I2df24323974ae0d3b3b0524240b64be5740977c6
Reviewed-on: https://chromium-review.googlesource.com/836760
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
This partially reverts commit d51ed57edb.
Reason for revert:
New git client for windows was rolled including fix for slow `git fetch`.
I guess smaller pack limit causes frequent bootstrap taking 2~3 minutes longer than
the case it does not happen.
Let me see what happen if we increase pack limit 9 -> 30.
I will increase this to 50 if this won't cause regression again.
Original change's description:
> git_cache: lower max num of .pack files before re-bootstrap on Win.
>
> It used to be 50, I think ~9 gives best results for Chromium on Win:
> on golo VM, it takes <4 minutes to re-boostrap + git fetch small
> delta, assuming zipped git checkout for bootstrap is fresh (~1day).
>
> For other repos, which are significantly smaller, this change should
> have minor effect if at all.
>
> Test: I tested this using `led` tool on Win7 machines running LUCI
> stack extensively. For example,
>
> * https://ci.chromium.org/swarming/task/3a0102e8c8657410
> shows case with few .pack files, hence just 1 fetch
>
> * https://ci.chromium.org/swarming/task/3a010282f9fd8010
> shows case with 39 .pack files and so bootstrapping + fetch.
> If you look at prior tasks on the same VM, you'd find this:
> https://ci.chromium.org/swarming/task/39ffe843d01ed010
> which spent 8 minutes doing 1 incremental fetch with 39 .pack
> files.
>
> **Troopers/Sheriffs**: This change is safe to revert.
> However, beware that you should also at the same time revert the recipe
> roll of this CL to the repo, in which the failed builder's recipe is
> located, typically `chromium/tools/build`.
>
> Bug: 749709
> Change-Id: I18e2b63283100d466e9fb981a9094862463f6909
> Reviewed-on: https://chromium-review.googlesource.com/787174
> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
> Reviewed-by: Takuto Ikuta <tikuta@google.com>
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 749709
Change-Id: I3052abe4a9b53277a60c0791a85355e7a0bbdf8f
Reviewed-on: https://chromium-review.googlesource.com/823544
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Takuto Ikuta <tikuta@google.com>
This will be used by various things in build and build_internal which
"need" a depot_tools on PATH (though I suspect most of those should be
refactored to use the depot_tools in chromium/src.git).
R=tandrii@chromium.org
Bug: 789808
Change-Id: I5d0590ccdc5482bd8e3048ad46a39b5aedacbddb
Reviewed-on: https://chromium-review.googlesource.com/828465
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Edward Lemur
Andrii Shyshkalov
Takuto Ikuta
agrieve
Stephen Martinis
Michael Moss
Ben Pastene
John Budorick
Robert Iannucci
Tom Anderson
Oleh Prypin
Takuto Ikuta
Aaron Gable
recipe-roller