Commit Graph

619 Commits (37f499daecbc6f9ce8e403914d2c29e59387e5b4)

Author SHA1 Message Date
Garrett Beaty 37f499daec Default to an empty footers dict instead of None.
By default during tests, the _get_footers_step call will return None for
the returned footers because it does not set any test data. This does
not match the actual behavior of the recipe because git_footers.py
returns an empty json object if the commit message has no footers. This
change updates it to return an empty dict instead of None and checks for
the cached footers value to be None instead of any non-true value so
that a commit message without footers doesn't get parsed multiple times.

Change-Id: I716a27e964eb92de138228df5cc0876322d82823
Recipe-Manual-Change: build
Recipe-Nontrivial-Roll: build_limited
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4492269
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2 years ago
Ariel Zhang 2d1c655c2c Add CL description getter
For context crrev.com/c/4450427.

R=gbeaty@google.com

Change-Id: Ic0a7847009be274010daeaf3181242ae34477a39
Bug: b/214597409
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4451469
Commit-Queue: Ariel Zhang <arielzhang@google.com>
Reviewed-by: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2 years ago
Joanna Wang fd9cea79c1 Use --upstream when calling bot_update (passed to gclient sync).
Should resolve this flaky issue in infra-try builds : https://ci.chromium.org/ui/p/infra/builders/try/infra-try-bionic-64/b8783448305870388449/overview

in logs we see

'''
________ running 'git reset --hard HEAD' in '/b/s/w/ir/cache/builder/infra'
[0:00:02] HEAD is now at 64b4e8f867 Committed patch
[0:00:03] Finished running: git -c core.quotePath=false status --porcelain
[0:00:03] Attempting rebase onto d94466a8462dd19c2a3fc0ca173fd1c89a1f3f28...
[0:00:04] Current branch tmp_checkin_branch is up to date.
[0:00:04] Finished running: git rev-parse HEAD
Checked out revision 64b4e8f867c7b8757438b7e64985654d80a3b135
'''
64b4e8f867 is not a valid infra/infra commit. It is a local commit created from the patch of the last try-job run.

Prior to switching recipes to infra_superproject, we have been depending on https://source.chromium.org/chromium/chromium/tools/depot_tools/+/main:recipes/recipe_modules/bot_update/resources/bot_update.py;l=753;drc=489304d94984bb006a03e842cf2c171c73e7d367 to bring the checkout back to a valid remote infra/infra commit before we get to `gclient sync`. But with infra_superproject as the new root solution, `git checkout --force <pin>` runs for infra_superproject.

'git reset --hard HEAD' is coming from:
https://source.chromium.org/chromium/chromium/tools/depot_tools/+/main:gclient_scm.py;l=786-791;drc=cf4658579326856b9b25cfe2efbfc8683af88814

This is noop. The previous patch commit IS HEAD.

With --upstream we get: `git reset --hard refs/remotes/origin/main`


Bug: 1421776, 1434522
Change-Id: I0d192a160fcc7c3d405765e2e03e05aeb60f6875
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4445126
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
2 years ago
Ben Pastene 97dadd025f Apply the gerrit REST timeout to only 'GetChanges' calls
https://crrev.com/c/4420526 put a 10s timeout on all calls made via
this script.

But it appears there are a variety of endpoints that take more than
10s, the latest being "SubmitChange", eg:
https://ci.chromium.org/ui/p/chromeos/builders/release/release-R113-15393.B-orchestrator/25/overview

So instead of applying the timeout to everything, and then opting
certain long-running calls out of it, this just opts-in the
"GetChanges" call to the timeout. This should cover the failure mode
that was originally solved for chrome's bots. And also bump the
timeout to 30s since we don't trust all get-changes queries to
reliably finish in under 10s.

This also bumps the step level timeout for the query in tryserver
recipe_mod to 8 min since there could be ~5min worth of sleeping +
150s worth of waiting+timing-out. So worst case the step will now
take 8min to fully exhaust all timeouts/sleeps.

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Bug: b/278083716
Change-Id: Ib366e004e0bb07297ba732590d488cae779e38ba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4426524
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
2 years ago
Ben Pastene 9519fc1300 Add timeouts to the actual http calls in gerrit_util.py
When gerrit's availability drops, chrome's builds see an excessive
amount of failures, eg:
https://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359780/overview
https://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359594/overview
https://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359564/overview
https://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359630/overview

Seemingly all failures occur in either the `gerrit fetch current CL
info` step or the `gerrit changes` step. Both steps have a 60s timeout
and shell out to depot_tools' gerrit_client.py. That script essentially
makes a single http call to gerrit. That request has no configured
timeout. So when gerrit's MIA and the call hangs indefinitely, so too
will the step hang. 60s after that, the step timeout is reached, and the
entire build crashes with an infra-failure.

However, one single retry has been shown to sufficiently work around
at least one instance of that failure:
dea9a6eca26f690ce10a959701a8fb51ad/+/build.proto

So this incorporates timeouts into the requests made by
gerrit_util.py. Each request is given a 10s timeout, which should be
enough for most/all gerrit calls. (Both steps have a p90 of less than
1sec.) When a timeout is reached, the script will automatically retry
up to 4 times.

This also bumps the timeouts of the step calls to gerrit_client.py to
6min since the script can now take up to 5 minutes to fail, ie:
the sequence of sleeps is roughly 10s, 20s, 40s, 80s, 160s, which is
about 5min. So a 6min timeout should cover all those retries.

This also passes in "--verbose" to all step calls to this script, so
the logging that prints out the retry + sleep log lines will be
visible in build logs.

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Bug: 1432638
Change-Id: I9dc47f4beeda3783ae4f9152bd29ee441ac3e197
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4420526
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
2 years ago
Joanna Wang 95a28c5a14 Update infra contexts to use infra_superproject.
Recipe-Nontrivial-Roll: infra
Bug: 1415507
Change-Id: I92a60640679115ae188c6de610ef695876a536b5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4378428
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Auto-Submit: Joanna Wang <jojwang@chromium.org>
2 years ago
Greg Edelston 02349da2cb recipe_modules/gsutil: Add `dry_run` kwarg
This kwarg will display the command that would have been run, without
actually running it. I plan to use this kwarg on some ChromeOS staging
builders.

BUG=b:270142110
TEST=./recipes.py test train

Change-Id: I20194152e305dc6bae71a58b94de71d414839643
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4382316
Reviewed-by: Gavin Mak <gavinmak@google.com>
Auto-Submit: Greg Edelston <gredelston@google.com>
Commit-Queue: Greg Edelston <gredelston@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
2 years ago
Joanna Wang 489304d949 Use os.renames() in remove() to auto-create subdirectories.
Bug: 1415507
Change-Id: Ie52123dc0844639116b55bc7f6a2734aaf7df33e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4378417
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Auto-Submit: Joanna Wang <jojwang@chromium.org>
2 years ago
Joanna Wang df528a9d7a Fix multiple endswith values.
Bug: 1421776
Change-Id: I6ca45a7aa086da4356302a7b9aa588bf60888129
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4375877
Reviewed-by: Gavin Mak <gavinmak@google.com>
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
2 years ago
Joanna Wang d9717259ee Adjust bot_update recipes to work with infra_superproject context.
Bug: 1415507
Change-Id: I0774eb4f2b7ea43b04a715c295f89e3adda6a9aa
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4347772
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
2 years ago
Joanna Wang d99bfd79b4 Add infra_superproject context.
Bug: 1415507
Change-Id: I89921959b6de164decd58aede3de7c8a6b504b7e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4368458
Commit-Queue: Yulan Lin <yulanlin@google.com>
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Yulan Lin <yulanlin@google.com>
2 years ago
Joanna Wang d04720454a Update recipes python tests.
For: http://shortn/_j5mixA1Mjr

Change-Id: I7ac4471fbb1e0629d59263266b3470e372d03a19
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4368938
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
2 years ago
Brian Sheedy 3971323b1d Remove get_footers patch_text TODO
Removes a TODO to remove the patch_text argument and its uses from
get_footers and related methods. The TODO indicated that it was
only used in example code, and thus should be handled another way.
However, it turns out that there is at least one legitimate use of
thus functionality downstream to retrieve footers from CL
descriptions earlier revisions.

Fixed: 1179039
Change-Id: I79996f6053a6769043943de39735b2b5d7ee02ee
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4283342
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
2 years ago
Aravind Vasudevan 214e1b269d Revert "Reset origin/HEAD when it points to master"
This reverts commit 19794fe91f.

Reason for revert: Doesn't fix the cache.

Original change's description:
> Reset origin/HEAD when it points to master
>
> This change tries resetting origin/HEAD when it points to master and the ref cannot be found. Post master->main migration, some cached repositories seem to point origin/HEAD to master.
>
> Bug: 1418866
> Change-Id: I83942901b105cace4d51deeac7e400f98d200168
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4296441
> Reviewed-by: Fumitoshi Ukai <ukai@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>

Bug: 1418866
Change-Id: I4c912f8401afb6fb0c2149c277778bdee9fa5d76
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4304240
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
2 years ago
Aravind Vasudevan 19794fe91f Reset origin/HEAD when it points to master
This change tries resetting origin/HEAD when it points to master and the ref cannot be found. Post master->main migration, some cached repositories seem to point origin/HEAD to master.

Bug: 1418866
Change-Id: I83942901b105cace4d51deeac7e400f98d200168
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4296441
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Brian Sheedy fce87337b1 Convert recipe_modules expectation tests
Converts all remaining recipe_modules expectation-based tests to be
assert/post_process-based.

Bug: 1418547
Change-Id: I0f45c92a4f682edcf186b5202ec2001475d04bec
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4288230
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
2 years ago
Brian Sheedy 4756d7c420 Add tryserver test_api method
Updates the tryserver recipe module's test_api to support overriding
get_footers data.

Bug: angleproject:7985
Change-Id: Ie571bc8d1c77cbb1cd4f18955b50d56e377ca903
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4278466
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Brian Sheedy <bsheedy@chromium.org>
2 years ago
chrome-bot cd2395991f Add crossbench to the gclient config
Bug:1410350
Change-Id: Iaf9fa01a92ed50c70224d480c3ee1b4acaf71aeb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4234360
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Ashwin Verleker <ashwinpv@google.com>
2 years ago
Junji Watanabe 9d77ca716f [recipe] Remove depot_tools.{ninja_path, autoninja_path} recipe APIs
I have replaced all the references, and there should be no references to these APIs anymore.

A PSA has been sent already.
http://g/chromium-dev-internal/n2IL0Mbh0E8

Bug: 1340825
Change-Id: Ife882f4427f8841cacb9a70a99dbd6a90c741da2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4194896
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2 years ago
Josip Sokcevic 884d716ce0 Run all presubmits with py3 if py2 isn't available
This utilizes env variable set by LUCI: https://crrev.com/c/4087937 and
luci.buildbucket.omit_python2 experiment.

R=gavinmak@google.com

Bug: 1401307
Change-Id: I0e6c6fae7cdd1e6044fe7d7c4008f7da3a1bbf23
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4179134
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2 years ago
Michael Achenbach 86cfa62b07 [presubmit] Use vpython3 on bots without vpython2
Bug: 1401307
Change-Id: Ie81608be8e8ec64ed569b0367f1426b4b2ab8b70
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4110968
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Stephen Martinis cf31045b34 Remove martiniss from OWNERS
I'm leaving Chrome

Change-Id: I6dad3a55a83de4a2609d316f23e82c8f2dc729a2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4117567
Auto-Submit: Stephen Martinis <martiniss@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
2 years ago
Josip Sokcevic 9d351f8b5a Revert "Update Presubmit to python3"
This reverts commit 1b7204d841.

Reason for revert: py2 presubmits no longer running, and some
aren't migrated to run py3 yet.

Original change's description:
> Update Presubmit to python3
>
> Recipe-Nontrivial-Roll: build
> Bug:1370463
> Change-Id: I0bf182badc16f0d75c7330f52906f4cf9ff66e08
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3975294
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Riley Wong <rgw@google.com>

Change-Id: I82c3f0f0d502ee896d4e79a6bb330aee71b4d309
Recipe-Nontrivial-Roll: build
Bug: 1370463
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4021307
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
3 years ago
Riley Wong 1b7204d841 Update Presubmit to python3
Recipe-Nontrivial-Roll: build
Bug:1370463
Change-Id: I0bf182badc16f0d75c7330f52906f4cf9ff66e08
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3975294
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Riley Wong <rgw@google.com>
3 years ago
Brian Sheedy 9879712089 Run recipe git_cl under Python 3
Makes the git_cl recipe module use vpython3 instead of vpython, as
the latter breaks when running on builders with the omit_python2
experiment.

Bug: 1366965
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Change-Id: I2306e97dbc7b75201024b5dcf1892787185d5f77
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3968801
Auto-Submit: Brian Sheedy <bsheedy@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
3 years ago
Riley Wong a662dd87fb Adding --no_diff flag. Needed to mute errors that are supposed to be
muted for --all.

Also adding back one level of verbose because now logs aren't
printing the stack trace for some errors.

Run Sample - https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket/8801029285027783553/+/u/presubmit_py3/json.output


Bug:1322936
Change-Id: Id1f30a5a8e7c5dc8a30af42e4b012f71554c340a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3946639
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Riley Wong <rgw@google.com>
3 years ago
Riley Wong 4e1c1958fa Removing log level info from Presubmit CI
STDOUT - https://ci.chromium.org/ui/p/chromium/builders/ci/linux-presubmit/8/overview has too much fluff

Bug:1322936
Change-Id: Ie516559e5006ae4dd613a82ac0ce6a705abf3820
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3933838
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Riley Wong <rgw@google.com>
3 years ago
Riley Wong ea876290e0 Accepting new flag for --all flag and allowing CI to run.
Bug:1322936
Change-Id: Idc829268aa11846325bd9db1b6b8d33094025f10
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3861697
Reviewed-by: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Riley Wong <rgw@google.com>
3 years ago
Garrett Beaty 3d1a52e074 Remove recipe warnings due to python module.
The python module is deprecated in favor of using the step module with
an explicit python binary. Switching the code to use the step module and
remove DEPS on recipe_engine/python removes warnings when running recipe
tests in the repo.

Change-Id: If3c9d5d6aff5aab69898b535399accbef92f5c0b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3880624
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
3 years ago
Zhaoyang Li 4858aea098 Roll mac_toolchain version in builders using osx_sdk
To take in change:

[mac_toolchain] Remove `sudo /usr/sbin/DevToolsSecurity -enable` | https://chromium-review.googlesource.com/c/infra/infra/+/3878350

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: infra
Bug: 1360487, b/244578145
Change-Id: Iff028748487e6ac8afa9739bfc6287b4c359bb5a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3879565
Reviewed-by: Garrett Beaty <gbeaty@google.com>
Commit-Queue: Zhaoyang Li <zhaoyangli@chromium.org>
3 years ago
Josip Sokcevic 228fedfaad Update diff_deps receipe to py3
This is a reland on 988c0af5be

R=aravindvasudev@google.com

Bug: 1357152
Recipe-Nontrivial-Roll: build
Change-Id: I93f8056908b50a7f41df488cfae200408d91e11b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3858159
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
3 years ago
Joanna Wang 96b57b9b59 [no-sync] bot_update: Remove previous no-sync exp changes and implement new one.
Bug: 1339472
Change-Id: I250dd72dd806d26ba6cbe213280ecd65fbc66123
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3789864
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
3 years ago
Takuto Ikuta 7298625c85 recipes: use python3 for git_footers.py
Bug: 1327534
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: infra
Change-Id: I5c84648eef8930f51d3809692d1cb3ffa46db6f9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3775846
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
3 years ago
Takuto Ikuta 90191b14c2 recipes: use vpython3 in gerrit recipe modules
This is to prevent failure like
https://ci.chromium.org/ui/p/chromium/builders/try/mac-rel/1057655/overview

Bug: 1327534
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Change-Id: I922dddbd23eae0a02a95e020722b5edd1186ef77
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3771540
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
3 years ago
Gavin Mak ea11cc31c5 Remove apply_patch_on_gclient property
This property isn't used anymore.

Bug: 891917
Change-Id: I005f5b320cb33afddb1d3272f3f626542996bb62
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3766100
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
3 years ago
Junji Watanabe 350d210871 Revert .cipd_bin setup in depot_tools recipe module
This reverts https://crrev.com/c/3697958 and https://crrev.com/c/3704921.
One reason is that recipe bundle is not designed to be mutated.
Another reason is the presence of gomacc binary under .cipd_bin causes an unexpected side effect. https://crbug.com/1337048

Bug: 931218, 1337048
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Change-Id: Iaea12a0ad9246c0781035aa499cc6c0bb42d71c3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3715873
Auto-Submit: Junji Watanabe <jwata@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
3 years ago
Junji Watanabe 93fbd989e0 Revert "Retry switching to use CIPD ninja v1.8.2"
This reverts commit 3133f98f3f.

Reason for revert: angle recipe fails to find third_party/depot_tool/ninja.exe
https://luci-milo.appspot.com/ui/p/angle/builders/try/win-trace/2061/overview

https://source.chromium.org/chromium/chromium/src/+/main:third_party/angle/src/tests/capture_replay_tests.py;l=134;drc=1d67951e0851d855c016c04cd5d11e10a0e12454

Original change's description:
> Retry switching to use CIPD ninja v1.8.2
>
> This reverts commit 0a5bae7ebc + some modifications on recipe module.
>
> The previous CL https://crrev.com/c/3697958 ensures depot_tools/.cipd_bin in recipe environments. This CL copies ninja.exe from .cipd_bin to depot_tools root as cipd_bin_setup.bat.
> For developers, it's expected to run update_depot_tools which triggers cipd_bin_setup.
> For CI/CQ, depot_tools._cipd_bin_setup() should be called before using ninja/autoninja recipe module APIs.
>
> Builds with depot_tools recipe bundle are tested using led.
>
> # Windows
> > led get-build 8811491780369249585 | led edit-recipe-bundle -O depot_tools="$HOME/depot_tools" | led launch
> https://ci.chromium.org/swarming/task/5b7512ee9d91fc10?
>
> # Linux
> > led get-build 8811504372914233697 | led edit-recipe-bundle -O depot_tools="$HOME/depot_tools" | led launch
> https://ci.chromium.org/swarming/task/5b7515832d0f7c10
>
> Original change's description:
> > Revert "Switch to use CIPD ninja v1.8.2"
> >
> > This reverts commit c90a982106.
> >
> > Reason for revert:
> > https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket/8811949917908303825/+/u/compile/stdout
> >
> > Original change's description:
> > > Switch to use CIPD ninja v1.8.2
> > >
> > > https://crrev.com/c/3674981 installed CIPD ninja packages.
> > > This CL switches from the old ninja binaries to them.
> > >
> > > Note that this CL doesn't change ninja version. the old/CIPD ninja binaries are v1.8.2.
> > >
> > > The differences from the old binaries are
> > > - the CIPD packages don't use chromium's clang.
> > > - the CIPD package for Linux doesn't use chromium's sysroot.
> > > - the min macOS version is 10.13, which was 10.6.
> > > Please also see https://crsrc.org/i/3pp/ninja/install_bootstrap.sh
> > >
> > >
> > > On Windows, ninja.exe will be installed under .cipd_bin.
> > > But there are many places that assume that ninja.exe exists on depot_tools root. So this CL also copies the ninja.exe from .cipd_bin to root in cipd_bin_setup.bat.
> > >
> > > Bug: 931218
> > > Change-Id: Ib67eee5e9b6ad9b2937b789626970d9c85867dbd
> > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3674982
> > > Commit-Queue: Junji Watanabe <jwata@google.com>
> > > Reviewed-by: Philipp Wollermann <philwo@google.com>
> > > Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
> > > Reviewed-by: Nico Weber <thakis@chromium.org>
> > > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> >
> > Bug: 931218
> > Change-Id: I08d61dad119a0d23ae9ec8b1ed787dd3915de697
> > No-Presubmit: true
> > No-Tree-Checks: true
> > No-Try: true
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3686854
> > Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
> > Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> > Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
> > Owners-Override: Takuto Ikuta <tikuta@chromium.org>
>
> Bug: 931218
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited
> Recipe-Nontrivial-Roll: chrome_release
> Recipe-Nontrivial-Roll: infra
> Change-Id: Ic7a9395c0e858823375f1a6ec79034dbc54230bb
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3696963
> Reviewed-by: Fumitoshi Ukai <ukai@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Junji Watanabe <jwata@google.com>
> Reviewed-by: Takuto Ikuta <tikuta@chromium.org>

Bug: 931218
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: infra
Change-Id: I6d44be002940331eb4b7cf88ca7aa1e717ba1514
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3708294
Auto-Submit: Junji Watanabe <jwata@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Owners-Override: Takuto Ikuta <tikuta@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
3 years ago
Junji Watanabe 7ff520f026 call _cipd_bin_setup() in depot_tools.on_path() recipe module API
Some users call ninja/autoninja inside a depot_tools.on_path() context.
e.g. https://source.chromium.org/chromium/chromium/tools/build/+/main:recipes/recipes/chromium_upload_clang.py;l=92;drc=5bf3de010da6bd44d776f98c136e3b41f82cdfe8

Tested on a led build.
https://ci.chromium.org/swarming/task/5b802dcdfe0a2810

Bug: 931218
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Change-Id: If3ece36c1d6590d30bc071deba6387d39b7c33a5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3704921
Reviewed-by: Alexander Thomas <athom@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Junji Watanabe <jwata@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Nico Weber <thakis@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
3 years ago
Junji Watanabe 3133f98f3f Retry switching to use CIPD ninja v1.8.2
This reverts commit 0a5bae7ebc + some modifications on recipe module.

The previous CL https://crrev.com/c/3697958 ensures depot_tools/.cipd_bin in recipe environments. This CL copies ninja.exe from .cipd_bin to depot_tools root as cipd_bin_setup.bat.
For developers, it's expected to run update_depot_tools which triggers cipd_bin_setup.
For CI/CQ, depot_tools._cipd_bin_setup() should be called before using ninja/autoninja recipe module APIs.

Builds with depot_tools recipe bundle are tested using led.

# Windows
> led get-build 8811491780369249585 | led edit-recipe-bundle -O depot_tools="$HOME/depot_tools" | led launch
https://ci.chromium.org/swarming/task/5b7512ee9d91fc10?

# Linux
> led get-build 8811504372914233697 | led edit-recipe-bundle -O depot_tools="$HOME/depot_tools" | led launch
https://ci.chromium.org/swarming/task/5b7515832d0f7c10

Original change's description:
> Revert "Switch to use CIPD ninja v1.8.2"
>
> This reverts commit c90a982106.
>
> Reason for revert:
> https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket/8811949917908303825/+/u/compile/stdout
>
> Original change's description:
> > Switch to use CIPD ninja v1.8.2
> >
> > https://crrev.com/c/3674981 installed CIPD ninja packages.
> > This CL switches from the old ninja binaries to them.
> >
> > Note that this CL doesn't change ninja version. the old/CIPD ninja binaries are v1.8.2.
> >
> > The differences from the old binaries are
> > - the CIPD packages don't use chromium's clang.
> > - the CIPD package for Linux doesn't use chromium's sysroot.
> > - the min macOS version is 10.13, which was 10.6.
> > Please also see https://crsrc.org/i/3pp/ninja/install_bootstrap.sh
> >
> >
> > On Windows, ninja.exe will be installed under .cipd_bin.
> > But there are many places that assume that ninja.exe exists on depot_tools root. So this CL also copies the ninja.exe from .cipd_bin to root in cipd_bin_setup.bat.
> >
> > Bug: 931218
> > Change-Id: Ib67eee5e9b6ad9b2937b789626970d9c85867dbd
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3674982
> > Commit-Queue: Junji Watanabe <jwata@google.com>
> > Reviewed-by: Philipp Wollermann <philwo@google.com>
> > Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
> > Reviewed-by: Nico Weber <thakis@chromium.org>
> > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
>
> Bug: 931218
> Change-Id: I08d61dad119a0d23ae9ec8b1ed787dd3915de697
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3686854
> Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
> Owners-Override: Takuto Ikuta <tikuta@chromium.org>

Bug: 931218
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: infra
Change-Id: Ic7a9395c0e858823375f1a6ec79034dbc54230bb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3696963
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
3 years ago
Junji Watanabe b6030901cf Setup cipd bin before calling ninja/autoninja
Tested in led builds on chromium/tools/build repo.

# Linux
```
> led get-build 8811504372914233697 | led edit-recipe-bundle -O depot_tools="$HOME/depot_tools" | led launch
...
https://ci.chromium.org/swarming/task/5b749884a3f3fa11
```

# Windows
```
> led get-build 8811491780369249585 | led edit-recipe-bundle -O https://ci.chromium.org/swarming/task/5b74bc67963d7610
```

Bug: 931218
Change-Id: I4a967f7544f71a9fef8a44baad45196365659596

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: infra
Change-Id: I4a967f7544f71a9fef8a44baad45196365659596
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3697958
Auto-Submit: Junji Watanabe <jwata@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
3 years ago
Ben Pastene 8d2d507a49 Add a method to gsutil recipe_module to disable multiprocessing on mac
In https://crbug.com/1327371, we started encountering hanging gsutil
calls. These were triggered under very specific circumstances, but
were essentially due to a multi-processing bug in python on MacOS:
https://bugs.python.org/issue33725

When running gsutil on mac, it explicitly suggests disabling
multi-processing for this reason:
https://source.chromium.org/chromium/chromium/src/+/main:third_party/catapult/third_party/gsutil/gslib/command.py;drc=3a12d6ccdec28da8bda09d9ff826aae1f9504e59;l=1331

So this CL simply puts that suggestion into practice. There are two
options for doing so: adding a line to an active Boto file, or adding
a "-o" arg to every gsutil invocation. Since chromium has multiple
copies of gsutil and multiple different locations where `gsutil` is
invoked, the latter option is considered intractable.

Instead, we choose the Boto file option by adding a context manager
to the gsutil recipe module that will insert a custom Boto file into
the environment that only disables multi-threading. Due to the fact
that the BOTO_PATH env var takes multiple paths, we can safely
incorporate both our custom Boto along with LUCI's per-build Boto file
in the same gsutil invocation. And when the context manager exits, the
environment should revert back to its original state.

Chromium incorporates this method in crrev.com/c/3662023. See it take
effect during the "gclient runhooks" step in this led build:
https://ci.chromium.org/swarming/task/5b0c5174e70cb010?server=chromium-swarm.appspot.com

Bug: 1327371
Change-Id: I75cfdf16d0ec023bf81ea57991dde996694a46c6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3661949
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
3 years ago
recipe-roller 4e6aa2530f Roll recipe dependencies (nontrivial).
This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.

The build that created this CL was
https://ci.chromium.org/b/8813625169557805905

Please review the expectation changes, and LGTM+CQ.

recipe_engine:
a58dd35fb7
  a58dd35 (tikuta@chromium.org)
      file: use vpython3 to run scripts in resources dir

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.

R=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Ignore-Freeze: Autoroller
Bugdroid-Send-Email: False
Change-Id: I116fa1e5faafade1c85a4cca7077b926af788a8d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3657049
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
3 years ago
Aravind Vasudevan 23ddab2235 Reland "Break make_encoded_file into two functions"
This is a reland of commit 5e49eda5c4

Original change's description:
> Break make_encoded_file into two functions
>
> The python3 fallback for `make_encoded_file()` produces different results on py2 and py3 when bytes are passed. This CL makes it clear breaking it into two methods, one for text and another for bytes. This CL also adds a fallback in `download_file` for py3 to return bytes when it cannot decode the downloaded file into utf-8.
>
> Recipe-Manual-Change: chromiumos
> Change-Id: I3d313e430c852e179825bc24bf4a58ce84440b2a
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3595026
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Reviewed-by: Greg Edelston <gredelston@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>

Recipe-Manual-Change: chromiumos
Change-Id: I321f797ae4a00deed7920ee6f80d666954b07c7d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3625907
Reviewed-by: Greg Edelston <gredelston@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
3 years ago
recipe-roller 5b13afcae3 Roll recipe dependencies (nontrivial).
This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.

The build that created this CL was
https://ci.chromium.org/b/8815259687168433617

Please review the expectation changes, and LGTM+CQ.

recipe_engine:
4cb1bf0fb0
  4cb1bf0 (gbeaty@chromium.org)
      Don't set the -u flag when running python unbuffered.

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.

R=iannucci@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Ignore-Freeze: Autoroller
Bugdroid-Send-Email: False
Change-Id: I0859db0f9db1c7548068073300150e979ac460b4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3621157
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Commit-Queue: Garrett Beaty <gbeaty@google.com>
Auto-Submit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Reviewed-by: Garrett Beaty <gbeaty@google.com>
3 years ago
Takuto Ikuta 8a87603683 Revert "Reland "Break make_encoded_file into two functions""
This reverts commit 41c57603f9.

Reason for revert:
This broke depot tools CQ.
e.g.
https://ci.chromium.org/ui/p/infra/builders/try/ChromiumOS%20downstream%20Recipe%20Roll%20tester%20from%20Depot%20Tools/b8815683872106857201/overview

Please prepare roll CL on chromeos side when you reland this.

Original change's description:
> Reland "Break make_encoded_file into two functions"
>
> This is a reland of commit 5e49eda5c4
>
> Original change's description:
> > Break make_encoded_file into two functions
> >
> > The python3 fallback for `make_encoded_file()` produces different results on py2 and py3 when bytes are passed. This CL makes it clear breaking it into two methods, one for text and another for bytes. This CL also adds a fallback in `download_file` for py3 to return bytes when it cannot decode the downloaded file into utf-8.
> >
> > Recipe-Manual-Change: chromiumos
> > Change-Id: I3d313e430c852e179825bc24bf4a58ce84440b2a
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3595026
> > Reviewed-by: Gavin Mak <gavinmak@google.com>
> > Reviewed-by: Greg Edelston <gredelston@google.com>
> > Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
>
> Recipe-Manual-Change: chromiumos
> Change-Id: I2de85eca71cc45f503c25888297ba881a5c662a8
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3610973
> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Greg Edelston <gredelston@google.com>

Change-Id: I7718ab01f4249c7dc2d978c12fb3a26e3ece91b6
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3612484
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
3 years ago
Aravind Vasudevan 41c57603f9 Reland "Break make_encoded_file into two functions"
This is a reland of commit 5e49eda5c4

Original change's description:
> Break make_encoded_file into two functions
>
> The python3 fallback for `make_encoded_file()` produces different results on py2 and py3 when bytes are passed. This CL makes it clear breaking it into two methods, one for text and another for bytes. This CL also adds a fallback in `download_file` for py3 to return bytes when it cannot decode the downloaded file into utf-8.
>
> Recipe-Manual-Change: chromiumos
> Change-Id: I3d313e430c852e179825bc24bf4a58ce84440b2a
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3595026
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Reviewed-by: Greg Edelston <gredelston@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>

Recipe-Manual-Change: chromiumos
Change-Id: I2de85eca71cc45f503c25888297ba881a5c662a8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3610973
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Greg Edelston <gredelston@google.com>
3 years ago
Ravi Mistry f7e510b2f7 Add download-topics support to bot_update
Tested by:
* `python recipes.py test run`
* Tested end-to-end by patching in this unsubmitted change in Skia's recipes with https://skia-review.googlesource.com/c/skia/+/532768

Bug: chromium:1319415
Change-Id: Ia1c9c495ef6482b3fdb494e1c1c9320541bcd0c5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3602901
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Ravi Mistry <rmistry@chromium.org>
3 years ago
Aravind Vasudevan ada0ebd47d Revert "Break make_encoded_file into two functions"
This reverts commit 5e49eda5c4.

Reason for revert: chromiumos rollout fails

Original change's description:
> Break make_encoded_file into two functions
>
> The python3 fallback for `make_encoded_file()` produces different results on py2 and py3 when bytes are passed. This CL makes it clear breaking it into two methods, one for text and another for bytes. This CL also adds a fallback in `download_file` for py3 to return bytes when it cannot decode the downloaded file into utf-8.
>
> Recipe-Manual-Change: chromiumos
> Change-Id: I3d313e430c852e179825bc24bf4a58ce84440b2a
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3595026
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Reviewed-by: Greg Edelston <gredelston@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>

Change-Id: Ibfb51614f9e19b575b99704aa4ca33c5525ab294
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3602168
Auto-Submit: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
3 years ago
Aravind Vasudevan 5e49eda5c4 Break make_encoded_file into two functions
The python3 fallback for `make_encoded_file()` produces different results on py2 and py3 when bytes are passed. This CL makes it clear breaking it into two methods, one for text and another for bytes. This CL also adds a fallback in `download_file` for py3 to return bytes when it cannot decode the downloaded file into utf-8.

Recipe-Manual-Change: chromiumos
Change-Id: I3d313e430c852e179825bc24bf4a58ce84440b2a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3595026
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Greg Edelston <gredelston@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
3 years ago
Garrett Beaty 3b97fa826e Reland "Set a default got_revision property in the bot_update json output."
This is a reland of commit 053817260a

The change was not related to the errors that it was reverted for, so no
changes are necessary.

Original change's description:
> Set a default got_revision property in the bot_update json output.
>
> The bot_update.py script sets a got_revision property by default even if
> it is not present in the reverse revision mapping. This results in an
> uncaught exception when set_output_commit is set to True if got_revision
> isn't present in the reverse revision mapping, but it isn't caught until
> production because the test API doesn't match that behavior. This change
> updates the test API method to match the behavior of the script.
>
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited
> Recipe-Nontrivial-Roll: chromiumos
> Recipe-Nontrivial-Roll: infra
> Change-Id: Ideefa9d77d2a816ae66a2bb52737264ed3f5bcee
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3575361
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Garrett Beaty <gbeaty@google.com>
> Auto-Submit: Garrett Beaty <gbeaty@google.com>

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Change-Id: I80efeee8a2b951df43b00d89bb596dc2cf2fcec7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3582182
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Garrett Beaty <gbeaty@google.com>
3 years ago