This is a reland of 856ccef0e9
Use a tempfile instead of stdout for gclient revinfo output
Original change's description:
> bot_update: Don't use gclient sync output and rely on gclient revinfo.
>
> In preparation for skipping gclient sync if there are no DEPS changes.
>
> Bug: 1199853
> Change-Id: Ib9b4ab803bc574a384c661765cee5e4c1de5baae
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2912259
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Bug: 1199853
Change-Id: Ib50c4e943cbb51e4a2463beee9b7d1dee824ad29
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2910950
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
The roll in depot_tools enables new features of uploading iOS runtimes
to separate packages, etc, for human users using the tool to upload
Xcode.
Bug: 1191260
Change-Id: Id6334afd599c9345f05c7fefbd8407f3bc3f8124
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2918693
Reviewed-by: Chan Li <chanli@chromium.org>
Commit-Queue: Zhaoyang Li <zhaoyangli@chromium.org>
This reverts commit 856ccef0e9.
Reason for revert: Probably broke chrome os waterfall
Original change's description:
> bot_update: Don't use gclient sync output and rely on gclient revinfo.
>
> In preparation for skipping gclient sync if there are no DEPS changes.
>
> Bug: 1199853
> Change-Id: Ib9b4ab803bc574a384c661765cee5e4c1de5baae
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2912259
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Bug: 1199853
Change-Id: Ib893704dffc68b02dfa7f977e751e236003b7e0b
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2910949
Auto-Submit: Stephen Martinis <martiniss@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Avi Drissman <avi@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
Owners-Override: Avi Drissman <avi@chromium.org>
In preparation for skipping gclient sync if there are no DEPS changes.
Bug: 1199853
Change-Id: Ib9b4ab803bc574a384c661765cee5e4c1de5baae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2912259
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Presubmits when uploading occasionally take a very long time - over a
minute - and after the fact there is no easy way to know why. This makes
fixes to slow presubmits take longer.
This change builds on crrev.com/c/2532895 to print a message when any
individual presubmit function takes longer than ten seconds. During
normal usage this is a NOP but it will presumably find the long poles
when presubmits are running particularly slowly.
This is similar to gclient runhooks where any hooks that take longer
than ten seconds have their run time printed.
Change-Id: If57ed35d7a7d221f6380e9b97cf72af56f75e441
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2911594
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
CheckForWindowsLineEndings is a Chromium presubmit to make sure that
\r\n line endings (Windows line endings) don't get committed. Among
other things these line endings mess up the license checks, leading to
cryptic errors.
The problem is that ChangedContents() was stripping line endings, making
any checking for them futile. This change adds an option to preserve
line endings, to be used by Chromium's CheckForWindowsLineEndings.
It's not clear how long this has been broken.
See also crrev.com/c/2911914 which must land after this.
Bug: 801033
Change-Id: I7cdb9b3581788624f9eca081da43bb070ee412a1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2910488
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
The CheckLicense presubmit is supposed to check for correct licenses,
not for correct line endings. By reading the input file in binary mode
it implictly does both, leading to confusing error messages.
This is exacerbated by bugs in the line ending checks which are being
addressed separately. This change will avoid false positives on files
that have Windows line endings.
Change-Id: I2ff8632f273ec4896cb4918c386e0d1c12e72935
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2910486
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.
recipe_engine:
f53df171ac
f53df17 (yiwzhang@google.com)
luciexe: don't clear tag in initial build when invoking sub-luc...
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
R=iannucci@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: Id9010b0da15d5ac9703abf3576e728b3abeaa164
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2911023
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Bot-Commit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.
recipe_engine:
e6c4e528a2
e6c4e52 (mgeorgaklis@google.com)
Added -merge flag functionality to resultdb query method.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
R=iannucci@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I2d952e5b82784cbdcb3274c11bbbf143420cb13d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2911020
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Bot-Commit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
This will attempt to do so in a Python2-compatible way.
Bug: 1210746
Change-Id: I09edc21a5c47106803c0ac5ca449e0f8732efb24
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2906501
Auto-Submit: Dirk Pranke <dpranke@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This reverts commit 68324eff2f.
Reason for revert: I think we need to revert this, because if someone tries to upload a CL from a revision older than when we switched to Python3, this'll run under Python2, where that parameter isn't accepted.
Sorry, I didn't think that part through.
Original change's description:
> Specify encoding for AUTHORS file
>
> Needed to resolve "UnicodeDecodeError" for Python 3 presubmit checks.
>
> Bug: 1210746
> Change-Id: If55190ffc9f99c671a84fe9b3539317e464639cc
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2904927
> Commit-Queue: Dirk Pranke <dpranke@google.com>
> Auto-Submit: Alan Screen <awscreen@chromium.org>
> Reviewed-by: Dirk Pranke <dpranke@google.com>
Bug: 1210746
Change-Id: I4e5a58869baa897066f4e84c2e18b673c25bcc34
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2906499
Auto-Submit: Dirk Pranke <dpranke@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
This reverts commit f059ec9368.
Reason for revert: Reverting because we probably need to be using vpython and a compiled crcmod instead. See, e.g.,. b/188591640.
Original change's description:
> Update gsutil to use gsutil version 4.61, python3.
>
> This CL updates the gsutil.py wrapper to download and use
> v4.61 of GCP's gsutil, which is Python3-compatible.
>
> v4.61 appears to be fully self-contained and have all of the
> packages it needs vendored into it. So, there's no reason to
> use vpython anymore, and this CL removes that.
>
> Also, this CL removes the 'fallback' option to gsutil and
> the ability to force a version switch, as this should no
> longer be necessary (it was added for a migration back in 2014
> but apparently this code was never removed afterwards).
>
> This CL also updates download_from_google_storage.py and
> upload_to_google_storage.py to similarly not have the version flags
> and to just use regular python3, not vpython3.
>
> Bug: 1184108
> Change-Id: I0d1a8351dba2d3ad1f927afa333fb10959f19443
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2898439
> Reviewed-by: Mike Frysinger <vapier@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Dirk Pranke <dpranke@google.com>
Bug: 1184108
Change-Id: I8e21a9a40d81e4e185642f866855b6838f80f1c2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2905904
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Dirk Pranke <dpranke@google.com>
Patch failures in bot_update return exit code 87 and 88, which are
ignored by ensure_checkout. This causes the bot_update step in recipes
to be successful despite a patch failure. This change removes these exit
codes from the accepted return codes for this step.
Bug: 1207685
Change-Id: I7bd71732a99184c96196659c6953e4d4b610c9b9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2893686
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
This fixes CL:2880843 to use a module already present in vpython.
Bug: 1206782
Change-Id: If4158a62011e043e89f40f368c9c644434df6614
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2901758
Commit-Queue: Allen Webb <allenwebb@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.
recipe_engine:
45b48dc4bf
45b48dc (hypan@google.com)
[cipd] Add `instances` support
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
R=iannucci@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I17a52c2fe983436c7e6363a0cd2852687d09156f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2900986
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Bot-Commit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
This CL updates the gsutil.py wrapper to download and use
v4.61 of GCP's gsutil, which is Python3-compatible.
v4.61 appears to be fully self-contained and have all of the
packages it needs vendored into it. So, there's no reason to
use vpython anymore, and this CL removes that.
Also, this CL removes the 'fallback' option to gsutil and
the ability to force a version switch, as this should no
longer be necessary (it was added for a migration back in 2014
but apparently this code was never removed afterwards).
This CL also updates download_from_google_storage.py and
upload_to_google_storage.py to similarly not have the version flags
and to just use regular python3, not vpython3.
Bug: 1184108
Change-Id: I0d1a8351dba2d3ad1f927afa333fb10959f19443
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2898439
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@google.com>
Enough works in that config that it seems like a better default by now.
Checking out, building, and uploading patches works (but uploading still
requires `--bypass-hooks` for now until crbug.com/1205263 is fixed).
(In case something important I'm not aware of breaks, we can revert this.)
Bug: 1103236,1205263
Change-Id: Id432ffd4ab19b15c019614953a2cefa787b46225
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2896468
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Nico Weber <thakis@chromium.org>
This is an automated CL created by the recipe roller. This CL rolls
recipe changes from upstream projects (recipe_engine) into this repository.
recipe_engine:
beaf875152
beaf875 (iannucci@chromium.org)
[python3] Switch all uses of `six` to `future`.
More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
R=iannucci@chromium.org
Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: Ic916edf96367712b8f3d455fa495f6268df01ec2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2897999
Commit-Queue: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
Bot-Commit: Recipe Roller <recipe-mega-autoroller@chops-service-accounts.iam.gserviceaccount.com>
This change includes the following commits from the infra/infra
repository:
$ git log --format='%h %s (%ae)' \
97ee9cecaac4d99be09278169f987e61b8116672..HEAD \
-- go/src/infra/chromeperf/pinpoint/cli
a16d4a5458 Fix deadlock in Pinpoint CLI Unit Test (heiserya@google.com)
211e030fbe Add ability to run a batch of experiments to Pinpoint CLI (heiserya@google.com)
ca40b45ace pinpoint: Make artifact download concurrent (dberris@google.com)
08ea607681 pinpoint: print absolute path for downloaded files (fancl@chromium.org)
128352a9fb chromeperf: Add go_package option to protos (dberris@google.com)
6f69cd0a52 pinpoint: Initial support for downloading job artifacts (fancl@chromium.org)
fbcd876092 pinpoint: Compact job listing UI (dberris@google.com)
095bfe3bc8 pinpoint: Set a user agent for the CLI (dberris@google.com)
Change-Id: I4c31857272591667af5587ec83f3418cb1c14303
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2896109
Auto-Submit: Chenlin Fan <fancl@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Dirk Pranke <dpranke@google.com>
Add revert instructions to .bat and bash files which are the entry
points to various infra tools deployed via CIPD.
Bug: 1206818
Change-Id: Ib0045aea86f29d0f7e3a43c6ff314c4b7aa74171
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2892387
Auto-Submit: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@google.com>
os.path.realpath will resolve back to the subst paths, leading to
inappropriate current working directories for all downstream hooks.
BUG=1208688
Change-Id: Idfcb0fe4d25ab1317db5aae34663b1017d4c4937
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2893669
Commit-Queue: Dirk Pranke <dpranke@google.com>
Auto-Submit: Robert Liao <robliao@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
I'm not saying this is why bot_update is failing, but I never trust python threads, especially not at scale. I also think the new implementation is easier to understand and less likely to have subtle mistakes if it is extended later
Change-Id: I29be55594a400fda845ea31893789104373fd0f6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2892185
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Anthony Polito <apolito@google.com>
This makes the file easier to read.
Change-Id: Ie5eac66582bd6f3ce3c31def6f591e001de9a79a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2893667
Auto-Submit: Robert Liao <robliao@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Edward Lesmes
Zhaoyang Li
Brian Ryner
Gavin Mak
Josip Sokcevic
Stephen Martinis
Fumitoshi Ukai
Nodir Turakulov
Bruce Dawson
Prakhar Asthana
recipe-roller
Dirk Pranke
Matt Kotsenas
Alan Screen
Vadim Shtayura
Allen Webb
Robert Iannucci
Erik Staab
Nico Weber
Evan Stade
Chenlin Fan
Robert Liao
Anthony Polito
Olivier Li