This is a reland of 72a245e4c6
The "git checkout master --detach" operation has been modified to run in the solution
directory.
Original change's description:
> Fetch the input commit ref when both ref and id are specified.
>
> This causes --refs refs/head/master to be added to bot_update calls when
> the input build message contains a commit id and the refs/head/master
> ref. This is an effective no-op.
>
> When bot_update clones a git repository, it results in HEAD referring to
> master in contrast to the non-clone case which results in a detached
> HEAD. This prevents fetching refs/heads/master, so detach the HEAD after
> cloning.
>
> Bug: 922150
> Change-Id: I357a78305020218e9f5d350832a706b8f2b27967
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Recipe-Nontrivial-Roll: infra
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1900735
> Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Bug: 922150
Change-Id: I7dc6ebed2dcbd880ff4d515f7e9fc6428ff24099
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: infra
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1910260
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
This reverts commit 72a245e4c6.
Reason for revert: crbug.com/1022617
Original change's description:
> Fetch the input commit ref when both ref and id are specified.
>
> This causes --refs refs/head/master to be added to bot_update calls when
> the input build message contains a commit id and the refs/head/master
> ref. This is an effective no-op.
>
> When bot_update clones a git repository, it results in HEAD referring to
> master in contrast to the non-clone case which results in a detached
> HEAD. This prevents fetching refs/heads/master, so detach the HEAD after
> cloning.
>
> Bug: 922150
> Change-Id: I357a78305020218e9f5d350832a706b8f2b27967
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Recipe-Nontrivial-Roll: infra
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1900735
> Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
TBR=ehmaldonado@chromium.org,gbeaty@chromium.org
Change-Id: Ibd6c6ed272b0c98f82be50cefa521aaee0444f11
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 922150, 1022617
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1904099
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
This causes --refs refs/head/master to be added to bot_update calls when
the input build message contains a commit id and the refs/head/master
ref. This is an effective no-op.
When bot_update clones a git repository, it results in HEAD referring to
master in contrast to the non-clone case which results in a detached
HEAD. This prevents fetching refs/heads/master, so detach the HEAD after
cloning.
Bug: 922150
Change-Id: I357a78305020218e9f5d350832a706b8f2b27967
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: infra
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1900735
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
There are 3 layers modified, starting from the bottom up:
1. git_cache.py populate
Now takes a --no-fetch-tags option. If specified, the cache will not
fetch updated tags from the server by passing --no-tags to git fetch.
This prevents the server from sending all the tag refs. In chromium
this prevents significant time bottlenecks dealing with 10k+ tags.
2. bot_update.py
bot_update has to deal with multiple git repos, it has the root repo
that is checked out through git-cache, as well as repos synched via
DEPS and gclient.
The script now takes a --no_fetch_tags option. If specified,
the git-cache populate call will include --no-fetch-tags. Otherwise, it
won't. So this controls (for chromium) if fetches to the src.git server
are passed --no-tags.
3. bot_update/api.py
This is the entry point for recipes and also has to deal with multiple
git repos. The behaviour at this point is not changed from the default.
A |no_fetch_tags| parameter is added to ensure_checkout() that defaults
to False.
This CL is a refactor with no intended behavior change.
The next step will be to change the chromium build repo to pass True
for ensure_checkout(no_fetch_tags) on chromium trybots.
This represents solution #2 in
https://docs.google.com/document/d/1hDIunJjjfpmr50y3YnZ4o3aq1NZF4gJa1TS0p7AIL64/edit#
Bug: 1019824
Change-Id: I935430603299daa9e301a95a5184c0ce486fd912
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1894352
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Erik Chen <erikchen@chromium.org>
Commit-Queue: danakj <danakj@chromium.org>
Auto-Submit: danakj <danakj@chromium.org>
This reverts commit 512f92b731.
Reason for revert: bot_update is having infra failure on ci bots.
For example https://ci.chromium.org/p/chromium/builders/ci/Closure%20Compilation%20Linux/249684.
Original change's description:
> Fetch the input commit ref when both ref and id are specified.
>
> This causes --refs refs/head/master to be added to bot_update
> calls when the input build message contains a commit id and the
> refs/head/master ref. This is an effective no-op.
>
> Bug: 922150
> Change-Id: I119d9b3cede727713a0b66bff83b5ee4e1848790
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1884457
> Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
> Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
TBR=tandrii@google.com,gbeaty@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 922150
Change-Id: Ie9275c21827688f59449f052c768452a8652e139
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1891000
Commit-Queue: Haiyang Pan <hypan@google.com>
Reviewed-by: Haiyang Pan <hypan@google.com>
This reverts commit 1e4dbf3f64.
Reason for revert: unlikely but possible cause of http://shortn/_sgL4PqICVB
Original change's description:
> Add more git tracing.
>
> This CL adds more perf tracing to `git fetch` from bot_update.py and
> git_cache.py
>
> Change-Id: I9b69c60b6c81fc0c5f23f82fcc889b4d45a27556
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: infra
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1888433
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
> Commit-Queue: Erik Chen <erikchen@chromium.org>
> Auto-Submit: Erik Chen <erikchen@chromium.org>
TBR=danakj@chromium.org,tandrii@google.com,erikchen@chromium.org
Change-Id: I03b33bf1532212f83a6b89b1de32fafb6b8aafc4
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1891001
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
This CL adds more perf tracing to `git fetch` from bot_update.py and
git_cache.py
Change-Id: I9b69c60b6c81fc0c5f23f82fcc889b4d45a27556
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1888433
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Erik Chen <erikchen@chromium.org>
Auto-Submit: Erik Chen <erikchen@chromium.org>
This causes --refs refs/head/master to be added to bot_update
calls when the input build message contains a commit id and the
refs/head/master ref. This is an effective no-op.
Bug: 922150
Change-Id: I119d9b3cede727713a0b66bff83b5ee4e1848790
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1884457
Commit-Queue: Garrett Beaty <gbeaty@chromium.org>
Auto-Submit: Garrett Beaty <gbeaty@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
This is a reland of e1410883a3
Use vpython to execute git_cl.py, gerrit_util.py and presubmit_support.py on recipes.
Original change's description:
> depot_tools: Add httplib2 to .vpython
>
> Check that things won't break before further changes are made.
>
> Bug: 1002153
> Change-Id: I41866f26334bf9ec2732bc0f25007234a95130e4
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1854749
> Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Bug: 1002153
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: release_scripts
Change-Id: Id94057eae8830dec197257df3ea35c0c4ff946b7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1856650
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Ran:
vi $(git grep --name-only iteritems | grep -v third_party)
vi $(git grep --name-only itervalues | grep -v third_party)
vi $(git grep --name-only 'print ' | grep -v third_party)
and edited the files quickly with adhoc macros. Then ran in recipes/:
./recipes.py test train
There was only a small subset of files that had been updated to use
six.iteritems() and six.itervalues(). Since the dataset size that is
being used in gclient is small (pretty much always below 200 items),
it's better to just switch to .items() right away and take the temporary
performance hit, so that we don't need to come back to rewrite the code.
Recipe-Nontrivial-Roll: build
Bug: 984182
Change-Id: I5faf11486b66b0d73c9098ab0f2ce1b15a45c53e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1854900
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Marc-Antoine Ruel <maruel@chromium.org>
This is a reland of 3f79763629.
Last time it broke a skia test. The test has since been fixed
in https://crbug.com/1005388
Original change's description:
> Set core.trustctime=false during git checkout operations.
>
> This should be faster and more general than running
> `git index --refresh` over all git repositories we can find.
>
> See "Proposal: Setting core.trustctime=false on Chromium's bots"
> on infra-dev (linked to from bug 330461 comment 45).
>
> Bug: 330461,642711,746763,740301.
> Change-Id: I7c35cffa107d46d9cf98f0e3e813ef2ab98df5fc
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1809419
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
> Commit-Queue: Nico Weber <thakis@chromium.org>
Recipe-Nontrivial-Roll: chromiumos
Bug: 330461, 642711, 746763, 740301.
Change-Id: I5d09a93689e53e4339bff7b1173df938aee73a4d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1810400
Commit-Queue: Nico Weber <thakis@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
This reverts commit 3f79763629.
Reason for revert:
Breaks Skia https://skia-review.googlesource.com/c/buildbot/+/242590
Original change's description:
> Set core.trustctime=false during git checkout operations.
>
> This should be faster and more general than running
> `git index --refresh` over all git repositories we can find.
>
> See "Proposal: Setting core.trustctime=false on Chromium's bots"
> on infra-dev (linked to from bug 330461 comment 45).
>
> Bug: 330461,642711,746763,740301.
> Change-Id: I7c35cffa107d46d9cf98f0e3e813ef2ab98df5fc
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1809419
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
> Commit-Queue: Nico Weber <thakis@chromium.org>
TBR=thakis@chromium.org,tandrii@google.com
Change-Id: I908ce92748f7b516efed5f2c02138ecfee306c88
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 330461, 642711, 746763, 740301.
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1811537
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Nico Weber <thakis@chromium.org>
This should be faster and more general than running
`git index --refresh` over all git repositories we can find.
See "Proposal: Setting core.trustctime=false on Chromium's bots"
on infra-dev (linked to from bug 330461 comment 45).
Bug: 330461,642711,746763,740301.
Change-Id: I7c35cffa107d46d9cf98f0e3e813ef2ab98df5fc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1809419
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Nico Weber <thakis@chromium.org>
In support of the signing rework we're going to want to have
the signers keep track of the calculated hash to ensure that
we don't have a TOCTOU type issue. This commit adds the stat
command to the gsutil module so that we can ask gs a files hash.
Change-Id: Ib35d1541292a18fbfc26fab50f594c682ba4109d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1754262
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: George Engelbrecht <engeg@google.com>
Mirroring crrev.com/c/1717663. bot_update runs that take more than an
hour are typically hung in irrecoverable ways.
Recipe-Nontrivial-Roll: build
Change-Id: Ic0afc48db7e0ac3464888e7715621a04f55443ce
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1754263
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Haiyang Pan <hypan@google.com>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
Auto-Submit: John Budorick <jbudorick@chromium.org>
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).
Please review the expectation changes, and LGTM+CQ.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
https://crrev.com/e7fec5ccb9e70bf17b7878e63052a874afa06856 Change is_luci default to True. (mmoss@google.com)
R=recipe-roller@chromium.org, tandrii@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I9eb8913a8f65b8f107e5856c46aceda10e14c00c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1752476
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
{_resolve_fixed_revisions} resolves every revision property. After a test
failure, we don't want to resolve every revision always.
This CL allows updating a subset of revisions for a new checkout.
R=machenbach@chromium.orgTBR=tandrii@chromium.org
Bug: 950731, 982711
Change-Id: I3cdc5cdb882009886b82557427fdf9b02e860b39
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1720929
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Auto-Submit: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Changed all headers in message to be the size of header 4 tags.
Made sure there is always a new line after code tag(```).
Change-Id: I772941a8bbf0634c9496ec648533110997e23cde
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1702293
Commit-Queue: Debrian Figueroa <debrian@google.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Added headers and code tags
Made sure that when limiting message size, at least part of the message
is seen.
Change-Id: I404e2f7e86907f701332a62fe46334ff437cca03
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1690741
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Debrian Figueroa <debrian@google.com>
Adds functionality to presubmit api to return error,
notification and warning messages caught in
presubmit_support.py in the form of a json.
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Bug:971895
Change-Id: I42a3df3994077342216d002381b6135012b4334c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1666250
Commit-Queue: Debrian Figueroa <debrian@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
To sanely support concurrency, I need to make `api.step.nest` strictly
scoped; i.e. once you leave the nest context it closes all the steps
within that context.
This changes some wild 'active_step' uses to generate a step for the
module's own purpose.
R=martiniss@chromium.org, nodir@chromium.org, tandrii@chromium.org
Recipe-Nontrivial-Roll: build
Bug: 910369
Change-Id: I19aef31118faf9d3d078a2c5a0ebd48a4bfebd43
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1630044
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Garrett Beaty
Stephen Martinis
danakj
Haiyang Pan
John Budorick
Erik Chen
Takuto Ikuta
Edward Lemur
Michael Achenbach
Nodir Turakulov
Marc-Antoine Ruel
Andrii Shyshkalov
Michael Moss
Nico Weber
Tamer Tas
recipe-roller
George Engelbrecht
James Darpinian
Debrian Figueroa
Erik Staab
Robert Iannucci