Update tryserver to use buildbucket.build.input.gerrit_changes

tryserver module's purpose is to provide high level API for patches.
Update it to use buildbucket.build.input.gerrit_changes[0].
Add helper properties for gerrit repo URL and gerrit ref.

This will be used by both bot_update and gclient.

R=hinoka@chromium.org, tandrii@chromium.org

Bug: 694348
Change-Id: I7eb416e75737310404c4354fa680fd3499eddaa7
Reviewed-on: https://chromium-review.googlesource.com/1211653
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>

recipes/README.recipes.md

@@ -761,17 +761,29 @@ Return a presubmit step.
 &emsp; **@property**<br>&mdash; **def [presubmit\_support\_path](/recipes/recipe_modules/presubmit/api.py#8)(self):**
 ### *recipe_modules* / [tryserver](/recipes/recipe_modules/tryserver)
 
-[DEPS](/recipes/recipe_modules/tryserver/__init__.py#5): [gerrit](#recipe_modules-gerrit), [git](#recipe_modules-git), [git\_cl](#recipe_modules-git_cl), [recipe\_engine/context][recipe_engine/recipe_modules/context], [recipe\_engine/json][recipe_engine/recipe_modules/json], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/platform][recipe_engine/recipe_modules/platform], [recipe\_engine/properties][recipe_engine/recipe_modules/properties], [recipe\_engine/python][recipe_engine/recipe_modules/python], [recipe\_engine/raw\_io][recipe_engine/recipe_modules/raw_io], [recipe\_engine/step][recipe_engine/recipe_modules/step]
+[DEPS](/recipes/recipe_modules/tryserver/__init__.py#5): [gerrit](#recipe_modules-gerrit), [git](#recipe_modules-git), [git\_cl](#recipe_modules-git_cl), [recipe\_engine/buildbucket][recipe_engine/recipe_modules/buildbucket], [recipe\_engine/context][recipe_engine/recipe_modules/context], [recipe\_engine/json][recipe_engine/recipe_modules/json], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/platform][recipe_engine/recipe_modules/platform], [recipe\_engine/properties][recipe_engine/recipe_modules/properties], [recipe\_engine/python][recipe_engine/recipe_modules/python], [recipe\_engine/raw\_io][recipe_engine/recipe_modules/raw_io], [recipe\_engine/step][recipe_engine/recipe_modules/step]
 
-#### **class [TryserverApi](/recipes/recipe_modules/tryserver/api.py#12)([RecipeApi][recipe_engine/wkt/RecipeApi]):**
+#### **class [TryserverApi](/recipes/recipe_modules/tryserver/api.py#11)([RecipeApi][recipe_engine/wkt/RecipeApi]):**
 
-&mdash; **def [add\_failure\_reason](/recipes/recipe_modules/tryserver/api.py#108)(self, reason):**
+&mdash; **def [add\_failure\_reason](/recipes/recipe_modules/tryserver/api.py#137)(self, reason):**
 
 Records a more detailed reason why build is failing.
 
 The reason can be any JSON-serializable object.
 
-&mdash; **def [get\_files\_affected\_by\_patch](/recipes/recipe_modules/tryserver/api.py#38)(self, patch_root, \*\*kwargs):**
+&emsp; **@property**<br>&mdash; **def [gerrit\_change](/recipes/recipe_modules/tryserver/api.py#30)(self):**
+
+Returns current gerrit change, if there is exactly one.
+
+Returns a self.m.buildbucket.common_pb2.GerritChange or None.
+
+&emsp; **@property**<br>&mdash; **def [gerrit\_change\_repo\_url](/recipes/recipe_modules/tryserver/api.py#38)(self):**
+
+Returns canonical URL of the gitiles repo of the current Gerrit CL.
+
+Populated iff gerrit_change is populated.
+
+&mdash; **def [get\_files\_affected\_by\_patch](/recipes/recipe_modules/tryserver/api.py#67)(self, patch_root, \*\*kwargs):**
 
 Returns list of paths to files affected by the patch.
 
@@ -781,34 +793,36 @@ Argument:
 
 Returned paths will be relative to to patch_root.
 
-&mdash; **def [get\_footer](/recipes/recipe_modules/tryserver/api.py#162)(self, tag, patch_text=None):**
+&mdash; **def [get\_footer](/recipes/recipe_modules/tryserver/api.py#192)(self, tag, patch_text=None):**
 
 Gets a specific tag from a CL description
 
-&mdash; **def [get\_footers](/recipes/recipe_modules/tryserver/api.py#144)(self, patch_text=None):**
+&mdash; **def [get\_footers](/recipes/recipe_modules/tryserver/api.py#173)(self, patch_text=None):**
 
 Retrieves footers from the patch description.
 
 footers are machine readable tags embedded in commit messages. See
 git-footers documentation for more information.
 
-&emsp; **@property**<br>&mdash; **def [is\_gerrit\_issue](/recipes/recipe_modules/tryserver/api.py#22)(self):**
+&mdash; **def [initialize](/recipes/recipe_modules/tryserver/api.py#19)(self):**
+
+&emsp; **@property**<br>&mdash; **def [is\_gerrit\_issue](/recipes/recipe_modules/tryserver/api.py#51)(self):**
 
 Returns true iff the properties exist to match a Gerrit issue.
 
-&emsp; **@property**<br>&mdash; **def [is\_patch\_in\_git](/recipes/recipe_modules/tryserver/api.py#32)(self):**
+&emsp; **@property**<br>&mdash; **def [is\_patch\_in\_git](/recipes/recipe_modules/tryserver/api.py#61)(self):**
 
-&emsp; **@property**<br>&mdash; **def [is\_tryserver](/recipes/recipe_modules/tryserver/api.py#17)(self):**
+&emsp; **@property**<br>&mdash; **def [is\_tryserver](/recipes/recipe_modules/tryserver/api.py#46)(self):**
 
 Returns true iff we have a change to check out.
 
-&mdash; **def [normalize\_footer\_name](/recipes/recipe_modules/tryserver/api.py#166)(self, footer):**
+&mdash; **def [normalize\_footer\_name](/recipes/recipe_modules/tryserver/api.py#196)(self, footer):**
 
-&mdash; **def [set\_compile\_failure\_tryjob\_result](/recipes/recipe_modules/tryserver/api.py#87)(self):**
+&mdash; **def [set\_compile\_failure\_tryjob\_result](/recipes/recipe_modules/tryserver/api.py#116)(self):**
 
 Mark the tryjob result as a compile failure.
 
-&emsp; **@contextlib.contextmanager**<br>&mdash; **def [set\_failure\_hash](/recipes/recipe_modules/tryserver/api.py#117)(self):**
+&emsp; **@contextlib.contextmanager**<br>&mdash; **def [set\_failure\_hash](/recipes/recipe_modules/tryserver/api.py#146)(self):**
 
 Context manager that sets a failure_hash build property on StepFailure.
 
@@ -817,7 +831,7 @@ for the same reason. For example, if a patch is bad (breaks something),
 we'd expect it to always break in the same way. Different failures
 for the same patch are usually a sign of flakiness.
 
-&mdash; **def [set\_invalid\_test\_results\_tryjob\_result](/recipes/recipe_modules/tryserver/api.py#99)(self):**
+&mdash; **def [set\_invalid\_test\_results\_tryjob\_result](/recipes/recipe_modules/tryserver/api.py#128)(self):**
 
 Mark the tryjob result as having invalid test results.
 
@@ -825,18 +839,18 @@ This means we run some tests, but the results were not valid
 (e.g. no list of specific test cases that failed, or too many
 tests failing, etc).
 
-&mdash; **def [set\_patch\_failure\_tryjob\_result](/recipes/recipe_modules/tryserver/api.py#83)(self):**
+&mdash; **def [set\_patch\_failure\_tryjob\_result](/recipes/recipe_modules/tryserver/api.py#112)(self):**
 
 Mark the tryjob result as failure to apply the patch.
 
-&mdash; **def [set\_subproject\_tag](/recipes/recipe_modules/tryserver/api.py#65)(self, subproject_tag):**
+&mdash; **def [set\_subproject\_tag](/recipes/recipe_modules/tryserver/api.py#94)(self, subproject_tag):**
 
 Adds a subproject tag to the build.
 
 This can be used to distinguish between builds that execute different steps
 depending on what was patched, e.g. blink vs. pure chromium patches.
 
-&mdash; **def [set\_test\_failure\_tryjob\_result](/recipes/recipe_modules/tryserver/api.py#91)(self):**
+&mdash; **def [set\_test\_failure\_tryjob\_result](/recipes/recipe_modules/tryserver/api.py#120)(self):**
 
 Mark the tryjob result as a test failure.
 
@@ -959,9 +973,9 @@ Move things around in a loop!
 &mdash; **def [RunSteps](/recipes/recipe_modules/presubmit/examples/full.py#10)(api):**
 ### *recipes* / [tryserver:examples/full](/recipes/recipe_modules/tryserver/examples/full.py)
 
-[DEPS](/recipes/recipe_modules/tryserver/examples/full.py#5): [tryserver](#recipe_modules-tryserver), [recipe\_engine/json][recipe_engine/recipe_modules/json], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/platform][recipe_engine/recipe_modules/platform], [recipe\_engine/properties][recipe_engine/recipe_modules/properties], [recipe\_engine/python][recipe_engine/recipe_modules/python], [recipe\_engine/raw\_io][recipe_engine/recipe_modules/raw_io], [recipe\_engine/step][recipe_engine/recipe_modules/step]
+[DEPS](/recipes/recipe_modules/tryserver/examples/full.py#5): [tryserver](#recipe_modules-tryserver), [recipe\_engine/buildbucket][recipe_engine/recipe_modules/buildbucket], [recipe\_engine/json][recipe_engine/recipe_modules/json], [recipe\_engine/path][recipe_engine/recipe_modules/path], [recipe\_engine/platform][recipe_engine/recipe_modules/platform], [recipe\_engine/properties][recipe_engine/recipe_modules/properties], [recipe\_engine/python][recipe_engine/recipe_modules/python], [recipe\_engine/raw\_io][recipe_engine/recipe_modules/raw_io], [recipe\_engine/step][recipe_engine/recipe_modules/step]
 
-&mdash; **def [RunSteps](/recipes/recipe_modules/tryserver/examples/full.py#17)(api):**
+&mdash; **def [RunSteps](/recipes/recipe_modules/tryserver/examples/full.py#18)(api):**
 ### *recipes* / [windows\_sdk:examples/full](/recipes/recipe_modules/windows_sdk/examples/full.py)
 
 [DEPS](/recipes/recipe_modules/windows_sdk/examples/full.py#5): [windows\_sdk](#recipe_modules-windows_sdk), [recipe\_engine/platform][recipe_engine/recipe_modules/platform], [recipe\_engine/properties][recipe_engine/recipe_modules/properties], [recipe\_engine/step][recipe_engine/recipe_modules/step]

recipes/recipe_modules/tryserver/__init__.py

@@ -6,6 +6,7 @@ DEPS = [
   'gerrit',
   'git',
   'git_cl',
+  'recipe_engine/buildbucket',
   'recipe_engine/context',
   'recipe_engine/json',
   'recipe_engine/path',

recipes/recipe_modules/tryserver/api.py

@@ -2,7 +2,6 @@
 # Use of this source code is governed by a BSD-style license that can be
 # found in the LICENSE file.
 
-import collections
 import contextlib
 import hashlib
 
@@ -14,6 +13,36 @@ class TryserverApi(recipe_api.RecipeApi):
     super(TryserverApi, self).__init__(*args, **kwargs)
     self._failure_reasons = []
 
+    self._gerrit_change = None  # self.m.buildbucket.common_pb2.GerritChange
+    self._gerrit_change_repo_url = None
+
+  def initialize(self):
+    changes = self.m.buildbucket.build.input.gerrit_changes
+    if len(changes) == 1:
+      cl = changes[0]
+      self._gerrit_change = cl
+      git_host = cl.host
+      gs_suffix = '-review.googlesource.com'
+      if git_host.endswith(gs_suffix):
+        git_host = '%s.googlesource.com' % git_host[:-len(gs_suffix)]
+      self._gerrit_change_repo_url = 'https://%s/%s' % (git_host, cl.project)
+
+  @property
+  def gerrit_change(self):
+    """Returns current gerrit change, if there is exactly one.
+
+    Returns a self.m.buildbucket.common_pb2.GerritChange or None.
+    """
+    return self._gerrit_change
+
+  @property
+  def gerrit_change_repo_url(self):
+    """Returns canonical URL of the gitiles repo of the current Gerrit CL.
+
+    Populated iff gerrit_change is populated.
+    """
+    return self._gerrit_change_repo_url
+
   @property
   def is_tryserver(self):
     """Returns true iff we have a change to check out."""
@@ -22,7 +51,7 @@ class TryserverApi(recipe_api.RecipeApi):
   @property
   def is_gerrit_issue(self):
     """Returns true iff the properties exist to match a Gerrit issue."""
-    if self.m.properties.get('patch_storage') == 'gerrit':
+    if self.gerrit_change:
       return True
     # TODO(tandrii): remove this, once nobody is using buildbot Gerrit Poller.
     return ('event.patchSet.ref' in self.m.properties and
@@ -148,10 +177,11 @@ class TryserverApi(recipe_api.RecipeApi):
     git-footers documentation for more information.
     """
     if patch_text is None:
+      if self.gerrit_change:
         patch_text = self.m.gerrit.get_change_description(
-          self.m.properties['patch_gerrit_url'],
+            'https://%s' % self.gerrit_change.host,
-          self.m.properties['patch_issue'],
+            int(self.gerrit_change.change),
-          self.m.properties['patch_set'])
+            int(self.gerrit_change.patchset))
 
     result = self.m.python(
         'parse description', self.package_repo_resource('git_footers.py'),

recipes/recipe_modules/tryserver/examples/full.expected/with_gerrit_patch.json

@@ -12,7 +12,7 @@
       "--limit",
       "1",
       "-p",
-      "change=456789",
+      "change=123456",
       "-o",
       "ALL_REVISIONS",
       "-o",
@@ -49,7 +49,7 @@
   },
   {
     "name": "$result",
-    "reason": "Error querying for CL description: host:'https://chromium-review.googlesource.com' change:456789; patchset:12",
+    "reason": "Error querying for CL description: host:u'https://chromium-review.googlesource.com' change:123456; patchset:7",
     "recipe_result": null,
     "status_code": 1
   }

recipes/recipe_modules/tryserver/examples/full.py

@@ -3,6 +3,7 @@
 # found in the LICENSE file.
 
 DEPS = [
+  'recipe_engine/buildbucket',
   'recipe_engine/json',
   'recipe_engine/raw_io',
   'recipe_engine/path',
@@ -28,6 +29,10 @@ def RunSteps(api):
             'Foo', api.properties['patch_text']))])
     return
 
+  if api.tryserver.gerrit_change:
+    assert (api.tryserver.gerrit_change_repo_url ==
+            'https://chromium.googlesource.com/chromium/src')
+
   if api.tryserver.is_gerrit_issue:
     api.tryserver.get_footers()
   api.tryserver.get_files_affected_by_patch(
@@ -75,7 +80,9 @@ def GenTests(api):
          api.properties(test_patch_root=''))
 
   yield (api.test('with_gerrit_patch') +
-         api.properties.tryserver(gerrit_project='infra/infra'))
+         api.buildbucket.try_build(
+            'chromium', 'linux',
+            git_repo='https://chromium.googlesource.com/chromium/src'))
 
   yield (api.test('with_wrong_patch_new') + api.platform('win', 32) +
          api.properties(test_patch_root='sub\\project'))