This is a reland of e1410883a3
Use vpython to execute git_cl.py, gerrit_util.py and presubmit_support.py on recipes.
Original change's description:
> depot_tools: Add httplib2 to .vpython
>
> Check that things won't break before further changes are made.
>
> Bug: 1002153
> Change-Id: I41866f26334bf9ec2732bc0f25007234a95130e4
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1854749
> Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Bug: 1002153
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: release_scripts
Change-Id: Id94057eae8830dec197257df3ea35c0c4ff946b7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1856650
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Ran:
vi $(git grep --name-only iteritems | grep -v third_party)
vi $(git grep --name-only itervalues | grep -v third_party)
vi $(git grep --name-only 'print ' | grep -v third_party)
and edited the files quickly with adhoc macros. Then ran in recipes/:
./recipes.py test train
There was only a small subset of files that had been updated to use
six.iteritems() and six.itervalues(). Since the dataset size that is
being used in gclient is small (pretty much always below 200 items),
it's better to just switch to .items() right away and take the temporary
performance hit, so that we don't need to come back to rewrite the code.
Recipe-Nontrivial-Roll: build
Bug: 984182
Change-Id: I5faf11486b66b0d73c9098ab0f2ce1b15a45c53e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1854900
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Marc-Antoine Ruel <maruel@chromium.org>
This is a reland of 3f79763629.
Last time it broke a skia test. The test has since been fixed
in https://crbug.com/1005388
Original change's description:
> Set core.trustctime=false during git checkout operations.
>
> This should be faster and more general than running
> `git index --refresh` over all git repositories we can find.
>
> See "Proposal: Setting core.trustctime=false on Chromium's bots"
> on infra-dev (linked to from bug 330461 comment 45).
>
> Bug: 330461,642711,746763,740301.
> Change-Id: I7c35cffa107d46d9cf98f0e3e813ef2ab98df5fc
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1809419
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
> Commit-Queue: Nico Weber <thakis@chromium.org>
Recipe-Nontrivial-Roll: chromiumos
Bug: 330461, 642711, 746763, 740301.
Change-Id: I5d09a93689e53e4339bff7b1173df938aee73a4d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1810400
Commit-Queue: Nico Weber <thakis@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
This reverts commit 3f79763629.
Reason for revert:
Breaks Skia https://skia-review.googlesource.com/c/buildbot/+/242590
Original change's description:
> Set core.trustctime=false during git checkout operations.
>
> This should be faster and more general than running
> `git index --refresh` over all git repositories we can find.
>
> See "Proposal: Setting core.trustctime=false on Chromium's bots"
> on infra-dev (linked to from bug 330461 comment 45).
>
> Bug: 330461,642711,746763,740301.
> Change-Id: I7c35cffa107d46d9cf98f0e3e813ef2ab98df5fc
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1809419
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
> Commit-Queue: Nico Weber <thakis@chromium.org>
TBR=thakis@chromium.org,tandrii@google.com
Change-Id: I908ce92748f7b516efed5f2c02138ecfee306c88
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 330461, 642711, 746763, 740301.
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1811537
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Nico Weber <thakis@chromium.org>
This should be faster and more general than running
`git index --refresh` over all git repositories we can find.
See "Proposal: Setting core.trustctime=false on Chromium's bots"
on infra-dev (linked to from bug 330461 comment 45).
Bug: 330461,642711,746763,740301.
Change-Id: I7c35cffa107d46d9cf98f0e3e813ef2ab98df5fc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1809419
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Nico Weber <thakis@chromium.org>
In support of the signing rework we're going to want to have
the signers keep track of the calculated hash to ensure that
we don't have a TOCTOU type issue. This commit adds the stat
command to the gsutil module so that we can ask gs a files hash.
Change-Id: Ib35d1541292a18fbfc26fab50f594c682ba4109d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1754262
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: George Engelbrecht <engeg@google.com>
Mirroring crrev.com/c/1717663. bot_update runs that take more than an
hour are typically hung in irrecoverable ways.
Recipe-Nontrivial-Roll: build
Change-Id: Ic0afc48db7e0ac3464888e7715621a04f55443ce
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1754263
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Haiyang Pan <hypan@google.com>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
Auto-Submit: John Budorick <jbudorick@chromium.org>
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).
Please review the expectation changes, and LGTM+CQ.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
https://crrev.com/e7fec5ccb9e70bf17b7878e63052a874afa06856 Change is_luci default to True. (mmoss@google.com)
R=recipe-roller@chromium.org, tandrii@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I9eb8913a8f65b8f107e5856c46aceda10e14c00c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1752476
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
{_resolve_fixed_revisions} resolves every revision property. After a test
failure, we don't want to resolve every revision always.
This CL allows updating a subset of revisions for a new checkout.
R=machenbach@chromium.orgTBR=tandrii@chromium.org
Bug: 950731, 982711
Change-Id: I3cdc5cdb882009886b82557427fdf9b02e860b39
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1720929
Commit-Queue: Tamer Tas <tmrts@chromium.org>
Auto-Submit: Tamer Tas <tmrts@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Changed all headers in message to be the size of header 4 tags.
Made sure there is always a new line after code tag(```).
Change-Id: I772941a8bbf0634c9496ec648533110997e23cde
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1702293
Commit-Queue: Debrian Figueroa <debrian@google.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Added headers and code tags
Made sure that when limiting message size, at least part of the message
is seen.
Change-Id: I404e2f7e86907f701332a62fe46334ff437cca03
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1690741
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Debrian Figueroa <debrian@google.com>
Adds functionality to presubmit api to return error,
notification and warning messages caught in
presubmit_support.py in the form of a json.
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Bug:971895
Change-Id: I42a3df3994077342216d002381b6135012b4334c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1666250
Commit-Queue: Debrian Figueroa <debrian@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
To sanely support concurrency, I need to make `api.step.nest` strictly
scoped; i.e. once you leave the nest context it closes all the steps
within that context.
This changes some wild 'active_step' uses to generate a step for the
module's own purpose.
R=martiniss@chromium.org, nodir@chromium.org, tandrii@chromium.org
Recipe-Nontrivial-Roll: build
Bug: 910369
Change-Id: I19aef31118faf9d3d078a2c5a0ebd48a4bfebd43
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1630044
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).
Please review the expectation changes, and LGTM+CQ.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
https://crrev.com/eeac8fccee349f073b6244a1032d083555b803ec Redact line numbers for traceback frames which originate outside the (iannucci@chromium.org)
R=recipe-roller@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: Iadda2159e026ed5e9e411dc4ecec2ff9cd36da32
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1620813
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
The repo it references is obsolete and I've been requested to delete it.
Bug: 963195
Change-Id: I4e765465950f85d4bdcb604804959ff29a00f062
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1611591
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
got_revision and got_revision_cp point to the commit that
Build.output.gitiles_commit should point to.
Therefore, derive output commit from these properties.
Derive output ref from got_revision_cp if it available.
Otherwise, try to read it from input commit
Bug: 940214
Change-Id: I108d782ad93c70114581830d7427b95d3efd9d98
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1588506
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
This change should hopefully enable the openscreen repo to use the
run_presubmit recipe.
Bug: openscreen:43
Change-Id: I672cddfeddd09ccc25ca851c0b613293191f17f9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1574189
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This reverts commit 9198ef8ede.
Reason for revert: Spec revert: appears to be causing large number of chromium CQ failures on patch application.
Original change's description:
> [bot_update] Default to refs/heads/master
>
> When revision is not specified, currently HEAD is checked out.
> Change the default to refs/heads/master
>
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Bug: 940214
> Change-Id: I8be59095d90447ea898d5247a6433f0734f5e63b
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1554862
> Commit-Queue: Nodir Turakulov <nodir@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
TBR=nodir@chromium.org,tandrii@chromium.org,jbudorick@chromium.org
Change-Id: Icafd817d8f044a4347342bd85dc97d193329a704
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 940214
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1555514
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
When revision is not specified, currently HEAD is checked out.
Change the default to refs/heads/master
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Bug: 940214
Change-Id: I8be59095d90447ea898d5247a6433f0734f5e63b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1554862
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This should allow for any recipe that uses the ANGLE repo to run
gclient sync with the right folder structure. Currently gclient sync
will check out third_party directories to the wrong folder.
Bug: angleproject:3288
Change-Id: Ia97592fdc0c9f3ec70c2d4a05d9934a6bc996507
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1551019
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Add set_output_commit param to bot_update.ensure_checkout, false by default.
If true, call api.buildbucket.set_output_gitiles_commit to set the output commit
of the build. This generally supersedes got_revision and got_revision_cp.
Bug: 940214
Change-Id: I5ffc790a6b12c8bf4a56469b3ecc567141c0d735
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1548256
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
This caused a bit of confusion for a customer who thought that they were
forbidden from passing Path objects in cmd.
R=tandrii@chromium.org
Change-Id: I352a6d5be75f48dcc0c8a8c92d6921359585ed48
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1536493
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Needed by modules using DIA SDK, which is located in "[windows_sdk]/DIA SDK",
not under "[windows_sdk]/win_sdk".
Change-Id: I1ca42a38ef8d9d1cbfde5e63e786b7d792504f3c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1526453
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Auto-Submit: Veranika Liaukevich <veranika@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
I've been using this recipe module for a new internal builder
(http://shortn/_wvjqgrpIGb) and it seems odd to have the package
name and instance ID in the step text rather than a link directly
to the site. (Perhaps the module predates the CIPD UI rewrite?)
Recipe-Nontrivial-Roll: infra
Change-Id: Iab6080c3c6110ac2205cb3de397055b30d881d62
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1496953
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
Noticed this while attempting to debug issues with the
src-internal-presubmit builder. Not sure if this is the cause but
would like to at least rule it out.
Bug: 925774
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: release_scripts
Change-Id: I323f49e23d2c8af11b374e22ebfbcf5bcd836ba3
Reviewed-on: https://chromium-review.googlesource.com/c/1464049
Commit-Queue: John Budorick <jbudorick@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
So that bot_update can apply patched on top of experimental branches,
which include the user's email (and so an extra @).
Change-Id: I33acb49e8b48c6ed6db7e752fca7eb0f1ede0690
Reviewed-on: https://chromium-review.googlesource.com/c/1450895
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Because of the (unfortunate) ways that PROPERTIES works currently,
setting defaults in the obvious way means that providing a subset of the
values as properties replaces the defaults entirely.
In this case we want to override the sdk_version without having to
override the toolchain app version.
R=jchinlee@chromium.org, tandrii@chromium.org
Change-Id: I13f4ee19e45a3bb147b37f69d10cb4ed4f734a21
Reviewed-on: https://chromium-review.googlesource.com/c/1438088
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).
Please review the expectation changes, and LGTM+CQ.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
https://crrev.com/3f3879c37865ac94e52e8d91a70399ca37ae4fb8 Make aggregated InfraFailures surface as exceptions. (joshuaseaton@google.com)
R=iannucci@chromium.org, martiniss@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I1ce8de0ea0660d734f553d18a051ba0552d263e5
Reviewed-on: https://chromium-review.googlesource.com/c/1382977
Commit-Queue: Recipe Roller <recipe-roller@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).
Please review the expectation changes, and LGTM+CQ.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
https://crrev.com/5e53d73e9bcec42830e10b835bc328eb75c9423f [path] Use 'cleanup' dir in mkdtemp and mkstemp (sergeyberezin@google.com)
R=iannucci@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I9a721c027e369ecd797e9fb1bfe707dfba15ed62
Reviewed-on: https://chromium-review.googlesource.com/c/1374435
Reviewed-by: Sergey Berezin <sergeyberezin@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Check that the cache contains the needed refs, and not only the revisions,
before attempting to checkout.
Bug: 910547
Change-Id: Ia269132421994803866065e81fc21ecd7ac206ed
Reviewed-on: https://chromium-review.googlesource.com/c/1362393
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).
Please review the expectation changes, and LGTM+CQ.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
https://crrev.com/faa8db3ceb17f3bc13c4ead7e4603760a0532f99 Use reason instead of reason_message. (jchinlee@chromium.org)
R=tandrii@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I3321c89b8fd9ace846de08149990cda5084ee78b
Reviewed-on: https://chromium-review.googlesource.com/c/1357323
Commit-Queue: Recipe Roller <recipe-roller@chromium.org>
Commit-Queue: Jao-ke Chin-Lee <jchinlee@chromium.org>
Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
The function can be called by chromium_test to set a property 'do_not_retry'
which will be propagated into buildbucket output.
Change-Id: I32d8ea925b7cb98d9b25d24226686e116c17801c
Bug: 910193
Reviewed-on: https://chromium-review.googlesource.com/c/1351542
Commit-Queue: Erik Chen <erikchen@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Ben Pastene <bpastene@chromium.org>
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).
Please review the expectation changes, and LGTM+CQ.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
https://crrev.com/793d8faa696f62fa2c774b8c854bbf4d64dff24e Separate InfraFailure behavior from StepFailure behavior. (jchinlee@chromium.org)
R=tandrii@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I396fcab3dbfcbf365d32483cf08f510805da303c
Reviewed-on: https://chromium-review.googlesource.com/c/1355801
Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
I'm writing a recipe that makes heavy use of gsutil's ls and cat, and
some of the calls are expected to fail. So this allows the user to make
those that fail do so as normal steps.
R=iannucci, tandrii
Bug: 866062
Change-Id: I4c9cdcb77c00afd4ab8fefd146f22fa94fcc37d7
Reviewed-on: https://chromium-review.googlesource.com/c/1325332
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Make autoninja path also available via depot_tools recipe module,
and make autoninja used where it is needed.
Bug: b/77176746
Change-Id: I8553cb3fb32f98e2e3ff420c255461b7e7da7df6
Reviewed-on: https://chromium-review.googlesource.com/c/1293112
Reviewed-by: Fumitoshi Ukai <ukai@chromium.org>
Reviewed-by: Shinya Kawanaka <shinyak@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Yoshisato Yanagisawa <yyanagisawa@chromium.org>
The LUCI environment alreadly does `git retry` style retries by virtue
of it's Go git-wrapper program. No need to put an extra python
interpreter in the middle.
R=hinoka@chromium.org, nodir@chromium.org
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Change-Id: I2918d7f413dde667fccd45c83ad6f4b96c2afe2e
Reviewed-on: https://chromium-review.googlesource.com/c/1292236
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Bug: chromium:892712
Change-Id: I68e1822a9a7205aaab3836be3f57a2f1feb1c9a6
Reviewed-on: https://chromium-review.googlesource.com/c/1289231
Commit-Queue: Scott Graham <scottmg@chromium.org>
Auto-Submit: Scott Graham <scottmg@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
This has been low-key driving me bananas for a while, and I've seen
it confuse sheriffs etc many times.
Change-Id: If46ab55821447a3a70a3f0ea774c017f29fc7400
Reviewed-on: https://chromium-review.googlesource.com/c/1286819
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
Some projects that use the git module (eg flutter) require tags.
Bug: 870558
Change-Id: Idae50ef5c8a092e5f0109b6c14c562268a2ad236
Reviewed-on: https://chromium-review.googlesource.com/c/1274245
Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
This should allow better integration with buildbucket named cache
support. Since this was already parameterized it shouldn't actually
break anything; previously the directory would be deleted on every run,
now it will still be deleted on every run (but we can add a buildbucket
named cache entry for it to preserve it).
R=tandrii@chromium.org, vadimsh@chromium.org
Bug: 877775
Change-Id: I1184fb59a3ed18bfda5fcb288903201b01527096
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Reviewed-on: https://chromium-review.googlesource.com/c/1263861
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
It has been True by default for a while, and there is no need to override it.
Bug: 891917
Change-Id: I2598a2230b0ea38a647a533757331c541b871971
Reviewed-on: https://chromium-review.googlesource.com/c/1260057
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
It looks like PNaCl actually depends on MSVC's presance, but doesn't
actually run any compile steps with it (instead using clang). That means
that mspdbsrv never actually runs during their build, and then the
taskkill fails.
R=hinoka@chromium.org, tandrii@chromium.org
Bug: 861512
Change-Id: I004d28f198224adaf16b1d3f14401b0d2a7d700b
Reviewed-on: https://chromium-review.googlesource.com/1232885
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
It is not used. get_gerrit_patch_root is used instead.
Bug: 694348
Change-Id: Icc717e38cbcaa8050c8ce8a364202b81d7f9cf2c
Reviewed-on: https://chromium-review.googlesource.com/1231282
Auto-Submit: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Use get_gerrit_patch_root in bot_update.ensure_checkout.
This fixes some tests expectations that expected CI builds to pass --patch_root
flag.
Also unify "root" and "patch_root" variables in ensure_checkout. Simpler.
This CL requires nontrivial rolls in all downstream packages because all
packages have incorrect expectations and/or tests.
In particular, this removes --patch_root flag from non-tryjob test
cases. Some tests are supposed to test tryjob scenario, but they use
properties.generic instead of properties.tryserver (or buildbucket.try_build)
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: release_scripts
Recipe-Nontrivial-Roll: skiabuildbot
Bug: 694348
Change-Id: Ibc594fa42e3b4d2823672b41dda2d97154b10d38
Reviewed-on: https://chromium-review.googlesource.com/1231280
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Update set_patch_project_revision to use repo_path_map instead of
patch_projects.
Also rename to set_patch_repo_revision.
Bug: 694348
Change-Id: Iba31f4910f8282160000b8fcef197052597530a4
Reviewed-on: https://chromium-review.googlesource.com/1226107
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Add a shortcut for getting local path to the patch repo.
In bot_update.api._destination_branch, start using it instead of reading
gclient config directly. This removes one more patch_project usage.
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Bug: 694348
Change-Id: If05371d0f6d11be6803ffa8392e029f5860dda25
Reviewed-on: https://chromium-review.googlesource.com/1217670
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Add tryserver.api.gerrit_change_info that returns detailed info about the
current Gerrit CL. It makes a Gerrit RPC in the background on the first read.
This step will be present in all tryjobs.
This mechanism can be used for extra details about the current CL.
This CL starts to use this in bot_update for gerrit_ref, and stops using
patch_ref property, and also update bot_update to use when the destination
branch is needed.
Also this CL removes no longer used properties in bot_update,
namely most of patch_* properties.
build.git manual roll: https://chromium-review.googlesource.com/c/chromium/tools/build/+/1217626
Recipe-Manual-Change: build
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: skiabuildbot
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: release_scripts
R=tandrii@chromium.org
Bug: 694348
Change-Id: I945d061b74278c3d1d227578abe16a64cdaca647
Reviewed-on: https://chromium-review.googlesource.com/1213557
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
CrOS chromium-pfq builder uses bot_update, but checks out Chromium on its own,
effectively ignoring input commit in bot_update.
To resolve a P0, add ignore_input_commit parameter.
Long term, bot_update API must be redesigned.
TBR=tandrii@chromium.org
Bug: 882094
Change-Id: I4f43348e3d5f81bf603dd2c110829bce58acad16
Reviewed-on: https://chromium-review.googlesource.com/1216787
Reviewed-by: Lann Martin <lannm@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
tryserver module's purpose is to provide high level API for patches.
Update it to use buildbucket.build.input.gerrit_changes[0].
Add helper properties for gerrit repo URL and gerrit ref.
This will be used by both bot_update and gclient.
R=hinoka@chromium.org, tandrii@chromium.org
Bug: 694348
Change-Id: I7eb416e75737310404c4354fa680fd3499eddaa7
Reviewed-on: https://chromium-review.googlesource.com/1211653
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Use self.m.buildbucket.build.input.gitiles_commit.id instead.
It encapsulates revision and parent_got_revision properties.
Also, honor host and project of the specified commit.
This enables overriding non-first gclient solution at the build-level
for arbitrary buildbucket builds.
Bug: 877161
Change-Id: I3f92f04ca84a1d30cbbb2670f62bea2bd9ee5932
Reviewed-on: https://chromium-review.googlesource.com/1200066
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Use repo_path_map in all places where patch_projects is currently used.
Add gclient.get_repo_path function to resolve path to a repo given its URL.
infra.git requires a non-trivial roll because of incorrect expectations.
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Bug: 877161
Change-Id: Iaea0a3f14609bf0c18d4ecdf8564153b8052584d
Reviewed-on: https://chromium-review.googlesource.com/1208363
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
When checking if buildbucket.build_input.gitiles_commit exists, use
HasField protobuf method, as opposed to truth check. A protobuf message
field is always true.
Bug: 877161
Change-Id: Ie4a7e22b985776441a74283b09646e8837c326f1
Reviewed-on: https://chromium-review.googlesource.com/1196930
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This is in order to use built-in presubit recipe.
R=maruel
Change-Id: I3e50af7dc3a9ad83f0f358285395794ea599a4e8
Reviewed-on: https://chromium-review.googlesource.com/1176650
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Also add shortcuts, so we don't have to look through all the branches
if the patch is based on a common branch (i.e. master, infra/config, lkgr)
Bug: 870279
Change-Id: I625a8481dccac9a475b096b926e6fab7efe676b0
Reviewed-on: https://chromium-review.googlesource.com/1161094
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).
Please review the expectation changes, and LGTM+CQ.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
https://crrev.com/bd5111dec043a367d69d85b81289e2207aa26f6e [cipd] Clean docs, pylint errors, add pkg-fetch, pkg-deploy. (iannucci@chromium.org)
R=tandrii@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I19d44529f7caa41c9577e57f6a9a35815627c4c3
Reviewed-on: https://chromium-review.googlesource.com/1170243
Commit-Queue: Recipe Roller <recipe-roller@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
When syncing remote branches, checkout origin/<branch> instead of
<branch>, since there might be a local branch with that name.
Bug: 859703
Change-Id: I9a49b940f53f75a97e5458fdcf578c3b52b26de2
Reviewed-on: https://chromium-review.googlesource.com/1128271
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Today, whenever LUCI scheduler sets gitiles_commit, a revision property
is also always set, thus this CL will have no effect on today's production.
The goal of CL is to pave the way to avoid this duplication. For starters,
one could start real writing tests with
+ api.buildbucket.ci_build(..., revision='sha1')
without the need to duplicate the same revision as a property. All
recipes which WAI with such input are thus ready to work w/o 'revision'
property. After that, we'll be able to stop the duplication in LUCI scheduler.
R=hinoka
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Change-Id: I1773b1c3d2a98a9dbd1cfdd9ac2b4fb99745ee99
Reviewed-on: https://chromium-review.googlesource.com/1123713
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
It is not possible to use cwd argument, since none of the methods called by this
module define cwd explicitly and low-level modules like api.step do not support
it anymore, resulting in a TypeError exception thrown due to unexpected keyword
argument, e.g. see https://ci.chromium.org/swarming/task/3e74a52905a71110. Given
that any users using cwd argument would be broken anyway, it should be safe to
make this change.
Planned usage: https://crrev.com/c/1122874
R=tandrii@chromium.org, iannucci@chromium.org
Bug: 793687
Change-Id: Iadf6bf91d8d41dbc6751a848bb69d7922a206ade
Reviewed-on: https://chromium-review.googlesource.com/1122873
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
gclient should be the one syncing the solutions to the appropriate
revisions.
Bug: 850812, 853032
Change-Id: Ieefc5661627d4864deb0d4e7053168a99da29d29
Reviewed-on: https://chromium-review.googlesource.com/1102833
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
shallow mode is a special git mode that only checks out a shallow copy
of git so the full history is not downloaded. It is not well supported
by googlesource.com servers because git has to do a giant negotiation
with the server and transmit every single commit that the local checkout
has or doesn't have (instead of just the latest commit of each ref),
which ends up taking more than 10x longer than a non-shallow checkout.
R=ehmaldonado, hinoka, iannucci
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Bug: 855137
Change-Id: I5f9e31f8b2730cf80b79bfd0e08201c33eb942f5
Reviewed-on: https://chromium-review.googlesource.com/1110450
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Allows easily getting at more than just the commit message.
I intend to look at the author to whitelist bots from binary size
checks (initially).
Bug: 702625
Change-Id: I684fdfacc88635ec4849216c74183f63f9d54810
Reviewed-on: https://chromium-review.googlesource.com/1070483
Commit-Queue: agrieve <agrieve@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
It fails when both the revision and the url are None.
Bug: 841936
Change-Id: Idef45a015624a92226d4ccd38ed5b978bf786993
Reviewed-on: https://chromium-review.googlesource.com/1053996
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Nodir Turakulov
Edward Lemur
Marc-Antoine Ruel
Andrii Shyshkalov
Michael Moss
Nico Weber
Tamer Tas
recipe-roller
George Engelbrecht
John Budorick
James Darpinian
Debrian Figueroa
Erik Staab
Robert Iannucci
Joey Scarr
Oleh Prypin
Stephen Martinis
Nodir Turakulov
Derek Schuff
btolsch
Jamie Madill
Takuto Ikuta
Veranika Liaukevich
Sergiy Belozorov
Andrii Shyshkalov
William Hesse
recipe-roller
erikchen
Ben Pastene
Mathieu Binette
Yoshisato Yanagisawa
Stephen Martinis
Scott Graham
Ryan Tseng
Corentin Wallez
Michael Achenbach
Andrew Grieve