Commit Graph

20 Commits (d51ed57edb595cd0e13e398925dba8bf2324e77e)

Author SHA1 Message Date
Andrii Shyshkalov 75c0b4b715 bot_update: don't crash badly if there no json outpot.
To illustrate, with this CL it's clear what is broken:
  https://ci.chromium.org/swarming/task/39e5676c13f00710?server=chromium-swarm.appspot.com
while before you get a red herring:
  ncaught Exception: AttributeError("'NoneType' object has no attribute
  'get'",)...
  https://ci.chromium.org/swarming/task/39e55d46c0774610?server=chromium-swarm.appspot.com

R=iannucci@chromium.org

Bug: 786486
Change-Id: I9ed2cdb7261c2e22a5a68116b81f841ece06c5d2
Reviewed-on: https://chromium-review.googlesource.com/777556
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Ryan Tseng bb85b3636b bot_update source manifest: fix test_api.py to match the actual output
The test output was incorrect, this matches what it actually should look like.

Bug: 772529
Change-Id: I1ca9ceeb6d6b6f55b2ac7c90667c6a9904d2a43b

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: release_scripts
Change-Id: I1ca9ceeb6d6b6f55b2ac7c90667c6a9904d2a43b
Reviewed-on: https://chromium-review.googlesource.com/751694
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
8 years ago
Ryan Tseng 70dea4270e Reland 2 "bot_update recipe: Upload source manifest"
This relands commit 47b67c426b.
    
Also:
* Fix bot_update.py output of manifest
    
Bug: 772529,776299
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: skiabuildbot
Recipe-Nontrivial-Roll: release_scripts
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: skiabuildbot
Recipe-Nontrivial-Roll: build

Change-Id: I31828ea5bfbc284623be8e553980f83d0889de92
Reviewed-on: https://chromium-review.googlesource.com/748308
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
8 years ago
Ryan Tseng 5d790d53ea Revert "Reland "bot_update recipe: Upload source manifest""
This reverts commit 47b67c426b.

Reason for revert: Bug in bot_update.py resource, breaks recipe roller.

Original change's description:
> Reland "bot_update recipe: Upload source manifest"
> 
> This reverts commit c3d1208d5c.
> 
> Also:
> * Instead of replacing "manifest", just add a new "source_manifest" 
> to the output JSON.  This allow transition without breakage.
> * Change the test api so test for recipe output changes.
> 
> The plan is to land this first, switch all downstream to "source_manifest",
> and then remove the original "manifest" key.
> 
> Bug: 772529,776299
> Change-Id: Iffb75f18046f8e4c058afe077872d4257b9dd754
> Recipe-Nontrivial-Roll: infra
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Recipe-Nontrivial-Roll: skiabuildbot
> Recipe-Nontrivial-Roll: release_scripts
> Recipe-Nontrivial-Roll: skia
> Recipe-Nontrivial-Roll: skiabuildbot
> Recipe-Nontrivial-Roll: build
> Reviewed-on: https://chromium-review.googlesource.com/731378
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Ryan Tseng <hinoka@chromium.org>

TBR=iannucci@chromium.org,hinoka@chromium.org

Change-Id: I7a4ee904075e8b75b8a47f9ef0cd8a633af85a9c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 772529, 776299
Reviewed-on: https://chromium-review.googlesource.com/748312
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
8 years ago
Ryan Tseng 47b67c426b Reland "bot_update recipe: Upload source manifest"
This reverts commit c3d1208d5c.

Also:
* Instead of replacing "manifest", just add a new "source_manifest" 
to the output JSON.  This allow transition without breakage.
* Change the test api so test for recipe output changes.

The plan is to land this first, switch all downstream to "source_manifest",
and then remove the original "manifest" key.

Bug: 772529,776299
Change-Id: Iffb75f18046f8e4c058afe077872d4257b9dd754
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: skiabuildbot
Recipe-Nontrivial-Roll: release_scripts
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: skiabuildbot
Recipe-Nontrivial-Roll: build
Reviewed-on: https://chromium-review.googlesource.com/731378
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
8 years ago
Michael Achenbach c3d1208d5c Revert "bot_update recipe: Upload source manifest"
This reverts commit 037f9b80bb.

Reason for revert: The structural change to manifest breaks the recipe
tester:
https://cs.chromium.org/chromium/infra/recipes/recipes/recipe_roll_tryjob.py?q=recipe_roll_tryjob.py&sq=package:chromium&dr&l=149
https://crbug.com/776299

Original change's description:
> bot_update recipe: Upload source manifest
> 
> Bug: 772529
> Change-Id: I3f833740ddfbc71be42b140c99dcd4bbfeed54fe
> Reviewed-on: https://chromium-review.googlesource.com/706071
> Commit-Queue: Ryan Tseng <hinoka@chromium.org>
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>

TBR=iannucci@chromium.org,hinoka@chromium.org,agable@chromium.org

Change-Id: Iba9653d73b7f117ca2f94619ae5da900cd0c8c98
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 772529,776299
Reviewed-on: https://chromium-review.googlesource.com/728059
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
8 years ago
Ryan Tseng 037f9b80bb bot_update recipe: Upload source manifest
Bug: 772529
Change-Id: I3f833740ddfbc71be42b140c99dcd4bbfeed54fe
Reviewed-on: https://chromium-review.googlesource.com/706071
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
8 years ago
Aaron Gable a16ccaf11a bot_update: use patch repo instead of project if it is mapped
This is a step away from patch_project, which was a holdover
from Rietveld. Instead of mapping "projects" (which are
poorly-defined and not guaranteed to be unique) to subpaths,
instead map repository urls (which are at least unique).

Eventually we may be able to compute this directly from DEPS
instead of hardcoding the mapping here, but at least this is
a step in the right direction.

Bug: 765633
Change-Id: Idd65984fc6edefcbedb0438d38c2338b10b7e8e5
Reviewed-on: https://chromium-review.googlesource.com/690776
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
8 years ago
Michael Achenbach c14b4f1518 Fix bot_update default value for empty revision
Bug: 753297
Change-Id: I31c031ce1efa3a1a7d30fd7eeb71d3e4b3baa720
Reviewed-on: https://chromium-review.googlesource.com/605609
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
8 years ago
Sergey Berezin 8038dfe3be bot_update recipe: parameterize step name in apply_gerrit_ref
Needed for parity with Rietveld two-sided patch application.

BUG=628259
R=agable@chromium.org

Change-Id: I79221f54e89a9291257a71b70cc52acc51167854
Reviewed-on: https://chromium-review.googlesource.com/591047
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Sergey Berezin <sergeyberezin@chromium.org>
8 years ago
Sergey Berezin 5aa5cd76f0 depot_tools: add gerrit repo and ref to apply_gerrit_ref()
The only user of this method needs to apply two patches from two
different repos in a try job. Extending the method to handle this.

BUG=628259
R=agable@chromium.org

Change-Id: Ia119780d1323add0094631e5c291b0e7a573e6e5
Reviewed-on: https://chromium-review.googlesource.com/588008
Commit-Queue: Sergey Berezin <sergeyberezin@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
8 years ago
Michael Achenbach 98681d113f Remove obsolete destination-branch-check flag
This was removed in downstream recipes after it was turned on by default.

Bug: 740456
Change-Id: Idae1905af97333970c8b31508634f8b71550c6f9
Reviewed-on: https://chromium-review.googlesource.com/574030
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
8 years ago
Michael Achenbach 8aa774cfb0 Fix bot_update example
This adds angle to the got_revision_mapping to take account of the checked
out revision and use it correctly in the without-patch application.

This just adds the got_revision property to all test cases in the output
and fixes the second bot_update run of two angle test cases.

Bug: 740456
Change-Id: I6ae6bf627653280bc2a94230ec77949411dc4331
Reviewed-on: https://chromium-review.googlesource.com/573741
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
8 years ago
Michael Achenbach f3d5a1147f Enable Gerrit-destination-branch check by default
Bug: 740456
Change-Id: Ib078d6aef7286bbe945513a5818e185d34f52058
Reviewed-on: https://chromium-review.googlesource.com/571802
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
8 years ago
Michael Achenbach 02e59414c1 Fix bot_update's usage of the destination branch
The previously passed branch:HEAD notation is understood by bot_update
for the main project, but not by gclient for deps'ed projects.

We don't need the colon notation at all as passing a branch implies
using the HEAD of that branch.

Bug: 740456
Change-Id: I95eb88f0de2e06bee8a3e7db24c4ad85cdb76d3e
Reviewed-on: https://chromium-review.googlesource.com/566866
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
8 years ago
Michael Achenbach 7dadf05dad Query Gerrit for destination branch in bot_update
This makes bot_update query Gerrit when syncing for Gerrit tryjobs.

The query will establish the actual destination branch of the CL,
which can be different from master (e.g. a feature branch).

Bot_update will ensure to use this destination branch for the repo
that corresponds to the CL's project. Both the main project or a
deps'ed project work.

Initially, this lives behind a flag that can be controlled in
downstream recipes. Eventually we'll set this to default after a
gradual roll-out.

Branches in branch-heads are not supported yet.

Bug: 740456
Change-Id: I4a0d50e2ca8fe90f8d29964a3ffab17291f7be60
Reviewed-on: https://chromium-review.googlesource.com/566824
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
8 years ago
Dan Jacques 70b3f3aae5 [bot_update] Pass spec as file.
"bot_update" currently passes a full "gclient" specification as a
command-line argument to "bot_update.py". This file contains all sorts
of junk that clutters the command-line, and revealed a batch file
escaping bug that seems to be present in the recipe engine. Change this
so that "bot_update" passes the spec as file content instead of a
command-line flag.

Also, "bot_update" apparently called "--spec", which optparse kindly
recognized as the real flag, "--specs". This has always(?) been a bug,
and is no longer a problem.

BUG=chromium:738348
TEST=expectations

Change-Id: I5324b94dd5ef78503e7cfdc28b5fde8f434e4359
Reviewed-on: https://chromium-review.googlesource.com/558207
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
8 years ago
Dan Jacques dc598e8223 [bot_update] Prefix PATH, cleanup.
Prefix "depot_tools" to PATH instead of appending. This will ensure that
"depot_tools" pulls resources from the recipe checkout as a preference,
not as a last resort.

Use "env_prefixes" instead of "env" to prefix PATH.

Remove Git speed limit settings, which are taken care of by the Git
wrapper now.

BUG=chromium:738176
TEST=expectations
R=hinoka@chromium.org, iannucci@chromium.org

Change-Id: I2e509a331266e867be323bc70e3bc0f08f85246f
Reviewed-on: https://chromium-review.googlesource.com/556287
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
8 years ago
Dan Jacques 59a74ed558 Add cleanup dir, use with bot_update.
Currently, "bot_update" relies on a BuildBot cleanup mechanism and, to a
lesser extent, the standard BuildBot directory layout. Both of these are
problematic when projecting it into other circumstances, notably
"remote_run" and LUCI.

Have "bot_update" handle its own cleanup. It will now choose a cleanup
directory within the hierarchy of its checkout, and explicitly purge it
prior to execution if it exists. This enforces its expected behavior in
all circumstances and removes its expectations of the greater checkout
layout.

Export "cleanup_dir" via "infra_paths" to point to "build.dead" when
running on BuildBot builds. Otherwise, it is a default directory which,
on Kitchen, is ephemeral by design.

BUG=chromium:725631
TEST=expectations

Change-Id: I664434c542a25aaa7ff3eac216208a2425730fde
Reviewed-on: https://chromium-review.googlesource.com/528057
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
8 years ago
Robert Iannucci 56038f517d rename example.py -> examples/full.py for all modules
R=dnj@chromium.org, nodir@chromium.org, phajdan.jr@chromium.org

Bug:
Change-Id: I47d8f9311b6ee71583fb989888a1e4d28b5495df
Reviewed-on: https://chromium-review.googlesource.com/520731
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
8 years ago