An out of date .boto file can prevent Google employees from being able
to download the Chromium toolchain from Google storage. An error message
was added to gsutil.py to help developers diagnose this, but the message
was swallowed by get_toolchain_if_necessary.py. This change prints the
error message, and adds '*' * 80 separators to make it more visible.
An obsolete import is also removed.
Bug: 1414152
Change-Id: I69d5a786672304ed363c291e0a8ef89f8be0f2e0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4310059
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
This change cleans up GSUTIL_ENABLE_LUCI_AUTH flag from gsutil. This is the final change to transition to luci-auth.
Fixed: 1359383
Change-Id: I1b4ac4bf95a887ebec92f71df343e855829846a4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4197458
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
gsutil.py adds a line break at the end of stdout/stderr when luci-auth is used. This change fixes it.
R=sokcevic, jojwang
Bug: 1412249, 1359383
Change-Id: Ieb200db94d9ed9149e17df93c33802a7433a8d5d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4221262
Auto-Submit: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
This change removes OOB deprecation log from gsutil. This is no longer necessary and it hinders with tools that use gsutil.py.
R=sokcevic, jojwang
Bug: 1412249, 1359383
Change-Id: If47239520a15d24658314bd44570babf99831e36
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4221926
Auto-Submit: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
This reland includes fix for crbug.com/1410703. It fallbacks to normal gsutil call when luci-auth is not logged in.
This is a reland of commit a7b20b34f8
Original change's description:
> Enable luci-auth by default
>
> This change enables luci-auth by default for gsutil.py, replacing OOB authentication flow.
>
> Bug: 1359383
> Change-Id: I1a93edc0d04e03c13921359205f252b9e1ed7594
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4193195
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Bug: 1359383
Change-Id: I9acfa8a578de8e106346b83aae509f9ffe1146e0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4199427
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
This is a reland of commit 5e5802d7a2
This was reverted as a temporary fix for servers running python 3.5. This reland removes `encoding` arg from `subprocess.run` temporarily until all servers are migrated to a supported python version.
Original change's description:
> Update log when luci-auth is not logged-in
>
> This change updates gsutil.py to capture when luci-auth is not logged in and prints useful log for signing in. This change couple enables luci-auth UI when luci-auth is enabled.
>
> Bug: 959170, 1359383
> Change-Id: Ic636bb362df388073e1fd11d736a47f35b749016
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4154550
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Bug: 959170, 1359383
Change-Id: I409a49362f0c584bd39f1031671e9e089cf50f97
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4165881
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
This reverts commit 5e5802d7a2.
Reason for revert: This CL introduces a dependency on Python version 3.10. I'm not clear that this is allowed for depot_tools. We are using it on systems with Python 3.5.
Original change's description:
> Update log when luci-auth is not logged-in
>
> This change updates gsutil.py to capture when luci-auth is not logged in and prints useful log for signing in. This change couple enables luci-auth UI when luci-auth is enabled.
>
> Bug: 959170, 1359383
> Change-Id: Ic636bb362df388073e1fd11d736a47f35b749016
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4154550
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Bug: 959170, 1359383
Change-Id: I0a73051f6b65786374f2b585f13ca88f5f8f83d5
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4165720
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
This change updates gsutil.py to capture when luci-auth is not logged in and prints useful log for signing in. This change couple enables luci-auth UI when luci-auth is enabled.
Bug: 959170, 1359383
Change-Id: Ic636bb362df388073e1fd11d736a47f35b749016
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4154550
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
The change updates gsutil.py to internally use luci-auth for
authentication. It redirects `gsutil.py config` to `luci-auth login` and
wraps gsutil.py calls using `luci-auth context`.
The change is mainly done to ensure users have a way to authenticate once OOB authentication flow is deprecated. This patch falls back to using the .boto file when that is provided.
Design Document: go/gsutil-oauth-issue-resolved
Bug: 959170, 1359383
Change-Id: Icc6df1d8b93424ac0f6e2df84afdebc869763ede
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4109192
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
This is a reland of commit 8edc960e9a
Original change's description:
> Update gsutil to use gsutil version 4.68
>
> We can't yet upgrade to version 5.x since it's only py3 compatbile and
> some builders still rely on running gsutil with py2.
>
> R=gavinmak@google.com
>
> Bug: 1184108
> Change-Id: I002cc3625a2184ab42383b88a014c0631d1a33ee
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3508516
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Bug: 1184108
Change-Id: I8f4b726c203ed12d415deeec249972fe855b59c8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3514940
Reviewed-by: Brian Ryner <bryner@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
This reverts commit 8edc960e9a.
Reason for revert: potentially breaking chromeos builders
Original change's description:
> Update gsutil to use gsutil version 4.68
>
> We can't yet upgrade to version 5.x since it's only py3 compatbile and
> some builders still rely on running gsutil with py2.
>
> R=gavinmak@google.com
>
> Bug: 1184108
> Change-Id: I002cc3625a2184ab42383b88a014c0631d1a33ee
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3508516
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Bug: 1184108
Change-Id: I40ce645ebd928c63116012353c632026c787b154
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3514508
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
We can't yet upgrade to version 5.x since it's only py3 compatbile and
some builders still rely on running gsutil with py2.
R=gavinmak@google.com
Bug: 1184108
Change-Id: I002cc3625a2184ab42383b88a014c0631d1a33ee
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3508516
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
This reverts commit e8ef6259b9.
Reason for revert: https://crbug.com/1240673
High rate of bot_update failures, lots of gsutil defunct
processes.
Original change's description:
> Update gsutil to use gsutil version 4.66, python3
>
> This is a reland of 457736028d
> with following changes:
> * bump version from 4.61 to 4.66, which contains several bugfixes
>
> Original change's description:
> > Reland "Reland "Update gsutil to use gsutil version 4.61, python3.""
> >
> > This is a reland of e53a593956
> >
> > Additional bug fixes:
> > * handle race condition in gsutil when creating its directory
> > * limit to one gsutil process on darwin due to bug in python3
> >
> > Original change's description:
> > > Reland "Update gsutil to use gsutil version 4.61, python3."
> > >
> > > This reverts commit af121aeec9.
> > >
> > > Reason for revert: re-landing with a switch back to vpython to get the compiled C extension version of crcmod for performance.
> > >
> > > Original change's description:
> > > > Revert "Update gsutil to use gsutil version 4.61, python3."
> > > >
> > > > This reverts commit f059ec9368.
> > > >
> > > > Reason for revert: Reverting because we probably need to be using vpython and a compiled crcmod instead. See, e.g.,. b/188591640.
> > > >
> > > > Original change's description:
> > > > > Update gsutil to use gsutil version 4.61, python3.
> > > > >
> > > > > This CL updates the gsutil.py wrapper to download and use
> > > > > v4.61 of GCP's gsutil, which is Python3-compatible.
> > > > >
> > > > > v4.61 appears to be fully self-contained and have all of the
> > > > > packages it needs vendored into it. So, there's no reason to
> > > > > use vpython anymore, and this CL removes that.
> > > > >
> > > > > Also, this CL removes the 'fallback' option to gsutil and
> > > > > the ability to force a version switch, as this should no
> > > > > longer be necessary (it was added for a migration back in 2014
> > > > > but apparently this code was never removed afterwards).
> > > > >
> > > > > This CL also updates download_from_google_storage.py and
> > > > > upload_to_google_storage.py to similarly not have the version flags
> > > > > and to just use regular python3, not vpython3.
> > > > >
> > > > > Bug: 1184108
> > > > > Change-Id: I0d1a8351dba2d3ad1f927afa333fb10959f19443
> > > > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2898439
> > > > > Reviewed-by: Mike Frysinger <vapier@chromium.org>
> > > > > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> > > > > Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> > > > > Commit-Queue: Dirk Pranke <dpranke@google.com>
> > > >
> > > > Bug: 1184108
> > > > Change-Id: I8e21a9a40d81e4e185642f866855b6838f80f1c2
> > > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2905904
> > > > Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> > > > Commit-Queue: Dirk Pranke <dpranke@google.com>
> > >
> > > Bug: 1184108
> > > Change-Id: I5d6d6d06842e08517488471c144972818fcf02ff
> > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2907155
> > > Reviewed-by: Mike Frysinger <vapier@chromium.org>
> > > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> > > Commit-Queue: Dirk Pranke <dpranke@google.com>
> >
> > Bug: 1184108
> > Change-Id: Ibb5d886fd22e3553521ff8ad6e2b4435844ef972
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2988716
> > Reviewed-by: Dirk Pranke <dpranke@google.com>
> > Reviewed-by: Mike Frysinger <vapier@chromium.org>
> > Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> > Commit-Queue: Josip Sokcevic <sokcevic@google.com>
>
> Bug: 1184108
> Change-Id: I33787dc75f6e45d6b462706e934d7a2a37703fa7
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3088085
> Reviewed-by: Dirk Pranke <dpranke@google.com>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Bug: 1184108, 1240673
Change-Id: I74c3243ea29b99476e09b6ddb49cb052812a1e3e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3100348
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
This is a reland of 457736028d
with following changes:
* bump version from 4.61 to 4.66, which contains several bugfixes
Original change's description:
> Reland "Reland "Update gsutil to use gsutil version 4.61, python3.""
>
> This is a reland of e53a593956
>
> Additional bug fixes:
> * handle race condition in gsutil when creating its directory
> * limit to one gsutil process on darwin due to bug in python3
>
> Original change's description:
> > Reland "Update gsutil to use gsutil version 4.61, python3."
> >
> > This reverts commit af121aeec9.
> >
> > Reason for revert: re-landing with a switch back to vpython to get the compiled C extension version of crcmod for performance.
> >
> > Original change's description:
> > > Revert "Update gsutil to use gsutil version 4.61, python3."
> > >
> > > This reverts commit f059ec9368.
> > >
> > > Reason for revert: Reverting because we probably need to be using vpython and a compiled crcmod instead. See, e.g.,. b/188591640.
> > >
> > > Original change's description:
> > > > Update gsutil to use gsutil version 4.61, python3.
> > > >
> > > > This CL updates the gsutil.py wrapper to download and use
> > > > v4.61 of GCP's gsutil, which is Python3-compatible.
> > > >
> > > > v4.61 appears to be fully self-contained and have all of the
> > > > packages it needs vendored into it. So, there's no reason to
> > > > use vpython anymore, and this CL removes that.
> > > >
> > > > Also, this CL removes the 'fallback' option to gsutil and
> > > > the ability to force a version switch, as this should no
> > > > longer be necessary (it was added for a migration back in 2014
> > > > but apparently this code was never removed afterwards).
> > > >
> > > > This CL also updates download_from_google_storage.py and
> > > > upload_to_google_storage.py to similarly not have the version flags
> > > > and to just use regular python3, not vpython3.
> > > >
> > > > Bug: 1184108
> > > > Change-Id: I0d1a8351dba2d3ad1f927afa333fb10959f19443
> > > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2898439
> > > > Reviewed-by: Mike Frysinger <vapier@chromium.org>
> > > > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> > > > Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> > > > Commit-Queue: Dirk Pranke <dpranke@google.com>
> > >
> > > Bug: 1184108
> > > Change-Id: I8e21a9a40d81e4e185642f866855b6838f80f1c2
> > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2905904
> > > Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> > > Commit-Queue: Dirk Pranke <dpranke@google.com>
> >
> > Bug: 1184108
> > Change-Id: I5d6d6d06842e08517488471c144972818fcf02ff
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2907155
> > Reviewed-by: Mike Frysinger <vapier@chromium.org>
> > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> > Commit-Queue: Dirk Pranke <dpranke@google.com>
>
> Bug: 1184108
> Change-Id: Ibb5d886fd22e3553521ff8ad6e2b4435844ef972
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2988716
> Reviewed-by: Dirk Pranke <dpranke@google.com>
> Reviewed-by: Mike Frysinger <vapier@chromium.org>
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Bug: 1184108
Change-Id: I33787dc75f6e45d6b462706e934d7a2a37703fa7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3088085
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
This reverts commit 457736028d.
Reason for revert: crbug/1227849 - android builders aren't happy,
also a report from user about android not working properly.
Original change's description:
> Reland "Reland "Update gsutil to use gsutil version 4.61, python3.""
>
> This is a reland of e53a593956
>
> Additional bug fixes:
> * handle race condition in gsutil when creating its directory
> * limit to one gsutil process on darwin due to bug in python3
>
> Original change's description:
> > Reland "Update gsutil to use gsutil version 4.61, python3."
> >
> > This reverts commit af121aeec9.
> >
> > Reason for revert: re-landing with a switch back to vpython to get the compiled C extension version of crcmod for performance.
> >
> > Original change's description:
> > > Revert "Update gsutil to use gsutil version 4.61, python3."
> > >
> > > This reverts commit f059ec9368.
> > >
> > > Reason for revert: Reverting because we probably need to be using vpython and a compiled crcmod instead. See, e.g.,. b/188591640.
> > >
> > > Original change's description:
> > > > Update gsutil to use gsutil version 4.61, python3.
> > > >
> > > > This CL updates the gsutil.py wrapper to download and use
> > > > v4.61 of GCP's gsutil, which is Python3-compatible.
> > > >
> > > > v4.61 appears to be fully self-contained and have all of the
> > > > packages it needs vendored into it. So, there's no reason to
> > > > use vpython anymore, and this CL removes that.
> > > >
> > > > Also, this CL removes the 'fallback' option to gsutil and
> > > > the ability to force a version switch, as this should no
> > > > longer be necessary (it was added for a migration back in 2014
> > > > but apparently this code was never removed afterwards).
> > > >
> > > > This CL also updates download_from_google_storage.py and
> > > > upload_to_google_storage.py to similarly not have the version flags
> > > > and to just use regular python3, not vpython3.
> > > >
> > > > Bug: 1184108
> > > > Change-Id: I0d1a8351dba2d3ad1f927afa333fb10959f19443
> > > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2898439
> > > > Reviewed-by: Mike Frysinger <vapier@chromium.org>
> > > > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> > > > Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> > > > Commit-Queue: Dirk Pranke <dpranke@google.com>
> > >
> > > Bug: 1184108
> > > Change-Id: I8e21a9a40d81e4e185642f866855b6838f80f1c2
> > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2905904
> > > Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> > > Commit-Queue: Dirk Pranke <dpranke@google.com>
> >
> > Bug: 1184108
> > Change-Id: I5d6d6d06842e08517488471c144972818fcf02ff
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2907155
> > Reviewed-by: Mike Frysinger <vapier@chromium.org>
> > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> > Commit-Queue: Dirk Pranke <dpranke@google.com>
>
> Bug: 1184108
> Change-Id: Ibb5d886fd22e3553521ff8ad6e2b4435844ef972
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2988716
> Reviewed-by: Dirk Pranke <dpranke@google.com>
> Reviewed-by: Mike Frysinger <vapier@chromium.org>
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Bug: 1184108
Bug: 1227849
Change-Id: Ib4176b94861e9f85caf6ed74050111a07b1e308d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3021641
Reviewed-by: Dirk Pranke <dpranke@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
This is a reland of e53a593956
Additional bug fixes:
* handle race condition in gsutil when creating its directory
* limit to one gsutil process on darwin due to bug in python3
Original change's description:
> Reland "Update gsutil to use gsutil version 4.61, python3."
>
> This reverts commit af121aeec9.
>
> Reason for revert: re-landing with a switch back to vpython to get the compiled C extension version of crcmod for performance.
>
> Original change's description:
> > Revert "Update gsutil to use gsutil version 4.61, python3."
> >
> > This reverts commit f059ec9368.
> >
> > Reason for revert: Reverting because we probably need to be using vpython and a compiled crcmod instead. See, e.g.,. b/188591640.
> >
> > Original change's description:
> > > Update gsutil to use gsutil version 4.61, python3.
> > >
> > > This CL updates the gsutil.py wrapper to download and use
> > > v4.61 of GCP's gsutil, which is Python3-compatible.
> > >
> > > v4.61 appears to be fully self-contained and have all of the
> > > packages it needs vendored into it. So, there's no reason to
> > > use vpython anymore, and this CL removes that.
> > >
> > > Also, this CL removes the 'fallback' option to gsutil and
> > > the ability to force a version switch, as this should no
> > > longer be necessary (it was added for a migration back in 2014
> > > but apparently this code was never removed afterwards).
> > >
> > > This CL also updates download_from_google_storage.py and
> > > upload_to_google_storage.py to similarly not have the version flags
> > > and to just use regular python3, not vpython3.
> > >
> > > Bug: 1184108
> > > Change-Id: I0d1a8351dba2d3ad1f927afa333fb10959f19443
> > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2898439
> > > Reviewed-by: Mike Frysinger <vapier@chromium.org>
> > > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> > > Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> > > Commit-Queue: Dirk Pranke <dpranke@google.com>
> >
> > Bug: 1184108
> > Change-Id: I8e21a9a40d81e4e185642f866855b6838f80f1c2
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2905904
> > Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> > Commit-Queue: Dirk Pranke <dpranke@google.com>
>
> Bug: 1184108
> Change-Id: I5d6d6d06842e08517488471c144972818fcf02ff
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2907155
> Reviewed-by: Mike Frysinger <vapier@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Dirk Pranke <dpranke@google.com>
Bug: 1184108
Change-Id: Ibb5d886fd22e3553521ff8ad6e2b4435844ef972
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2988716
Reviewed-by: Dirk Pranke <dpranke@google.com>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
This reverts commit e53a593956.
Reason for revert: Speculative revert for build failures on Mac11 bots: crbug.com/1219065.
Original change's description:
> Reland "Update gsutil to use gsutil version 4.61, python3."
>
> This reverts commit af121aeec9.
>
> Reason for revert: re-landing with a switch back to vpython to get the compiled C extension version of crcmod for performance.
>
> Original change's description:
> > Revert "Update gsutil to use gsutil version 4.61, python3."
> >
> > This reverts commit f059ec9368.
> >
> > Reason for revert: Reverting because we probably need to be using vpython and a compiled crcmod instead. See, e.g.,. b/188591640.
> >
> > Original change's description:
> > > Update gsutil to use gsutil version 4.61, python3.
> > >
> > > This CL updates the gsutil.py wrapper to download and use
> > > v4.61 of GCP's gsutil, which is Python3-compatible.
> > >
> > > v4.61 appears to be fully self-contained and have all of the
> > > packages it needs vendored into it. So, there's no reason to
> > > use vpython anymore, and this CL removes that.
> > >
> > > Also, this CL removes the 'fallback' option to gsutil and
> > > the ability to force a version switch, as this should no
> > > longer be necessary (it was added for a migration back in 2014
> > > but apparently this code was never removed afterwards).
> > >
> > > This CL also updates download_from_google_storage.py and
> > > upload_to_google_storage.py to similarly not have the version flags
> > > and to just use regular python3, not vpython3.
> > >
> > > Bug: 1184108
> > > Change-Id: I0d1a8351dba2d3ad1f927afa333fb10959f19443
> > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2898439
> > > Reviewed-by: Mike Frysinger <vapier@chromium.org>
> > > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> > > Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> > > Commit-Queue: Dirk Pranke <dpranke@google.com>
> >
> > Bug: 1184108
> > Change-Id: I8e21a9a40d81e4e185642f866855b6838f80f1c2
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2905904
> > Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> > Commit-Queue: Dirk Pranke <dpranke@google.com>
>
> Bug: 1184108
> Change-Id: I5d6d6d06842e08517488471c144972818fcf02ff
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2907155
> Reviewed-by: Mike Frysinger <vapier@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Dirk Pranke <dpranke@google.com>
Bug: 1184108
Change-Id: Ib0757e0873d6e6b39621d50d327d959f0fa43720
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2957473
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Dirk Pranke <dpranke@google.com>
This reverts commit af121aeec9.
Reason for revert: re-landing with a switch back to vpython to get the compiled C extension version of crcmod for performance.
Original change's description:
> Revert "Update gsutil to use gsutil version 4.61, python3."
>
> This reverts commit f059ec9368.
>
> Reason for revert: Reverting because we probably need to be using vpython and a compiled crcmod instead. See, e.g.,. b/188591640.
>
> Original change's description:
> > Update gsutil to use gsutil version 4.61, python3.
> >
> > This CL updates the gsutil.py wrapper to download and use
> > v4.61 of GCP's gsutil, which is Python3-compatible.
> >
> > v4.61 appears to be fully self-contained and have all of the
> > packages it needs vendored into it. So, there's no reason to
> > use vpython anymore, and this CL removes that.
> >
> > Also, this CL removes the 'fallback' option to gsutil and
> > the ability to force a version switch, as this should no
> > longer be necessary (it was added for a migration back in 2014
> > but apparently this code was never removed afterwards).
> >
> > This CL also updates download_from_google_storage.py and
> > upload_to_google_storage.py to similarly not have the version flags
> > and to just use regular python3, not vpython3.
> >
> > Bug: 1184108
> > Change-Id: I0d1a8351dba2d3ad1f927afa333fb10959f19443
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2898439
> > Reviewed-by: Mike Frysinger <vapier@chromium.org>
> > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> > Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> > Commit-Queue: Dirk Pranke <dpranke@google.com>
>
> Bug: 1184108
> Change-Id: I8e21a9a40d81e4e185642f866855b6838f80f1c2
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2905904
> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> Commit-Queue: Dirk Pranke <dpranke@google.com>
Bug: 1184108
Change-Id: I5d6d6d06842e08517488471c144972818fcf02ff
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2907155
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Dirk Pranke <dpranke@google.com>
This reverts commit f059ec9368.
Reason for revert: Reverting because we probably need to be using vpython and a compiled crcmod instead. See, e.g.,. b/188591640.
Original change's description:
> Update gsutil to use gsutil version 4.61, python3.
>
> This CL updates the gsutil.py wrapper to download and use
> v4.61 of GCP's gsutil, which is Python3-compatible.
>
> v4.61 appears to be fully self-contained and have all of the
> packages it needs vendored into it. So, there's no reason to
> use vpython anymore, and this CL removes that.
>
> Also, this CL removes the 'fallback' option to gsutil and
> the ability to force a version switch, as this should no
> longer be necessary (it was added for a migration back in 2014
> but apparently this code was never removed afterwards).
>
> This CL also updates download_from_google_storage.py and
> upload_to_google_storage.py to similarly not have the version flags
> and to just use regular python3, not vpython3.
>
> Bug: 1184108
> Change-Id: I0d1a8351dba2d3ad1f927afa333fb10959f19443
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2898439
> Reviewed-by: Mike Frysinger <vapier@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Dirk Pranke <dpranke@google.com>
Bug: 1184108
Change-Id: I8e21a9a40d81e4e185642f866855b6838f80f1c2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2905904
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Dirk Pranke <dpranke@google.com>
This CL updates the gsutil.py wrapper to download and use
v4.61 of GCP's gsutil, which is Python3-compatible.
v4.61 appears to be fully self-contained and have all of the
packages it needs vendored into it. So, there's no reason to
use vpython anymore, and this CL removes that.
Also, this CL removes the 'fallback' option to gsutil and
the ability to force a version switch, as this should no
longer be necessary (it was added for a migration back in 2014
but apparently this code was never removed afterwards).
This CL also updates download_from_google_storage.py and
upload_to_google_storage.py to similarly not have the version flags
and to just use regular python3, not vpython3.
Bug: 1184108
Change-Id: I0d1a8351dba2d3ad1f927afa333fb10959f19443
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2898439
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@google.com>
This enables gclient sync and gclient runhooks to run, barring hook script failures.
git cl upload also now works.
The scripts still work with Python 2.
There are no intended behaviour changes.
Bug: 942522
Change-Id: I2ac587b5f803ba7f5bb5e412337ce049f4b1a741
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1524583
Commit-Queue: Raul Tambre <raul@tambre.ee>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Picks up about 2 years worth of changes. Should be relatively risk-free since
we pin gsutil.py in almost all of our scripts.
Bug:
Change-Id: Iff1365ee6d6f31e03da8d8876b6ccd0b47a033c2
Reviewed-on: https://chromium-review.googlesource.com/801355
Reviewed-by: Erik Staab <estaab@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Jao-ke Chin-Lee <jchinlee@chromium.org>
The vpython/CIPD work created some errors when attempting to build on
Cygwin. This patch fixes these errors by converting Cygwin paths to
Windows paths or using the depot_tools python instead of Cygwin python.
Bug:
Change-Id: Id62f59a357fa2937c7eb843bf306ef4303b0910e
Reviewed-on: https://chromium-review.googlesource.com/745727
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
the tests haven't been ran by presumbit for a while because of the plural in
the filename.
At some point some post "gsutil cp" file checking happened, which
broke the tests. This adds a callback to the fake gsutil cp so
that the expect file is copied over.
This also removes "gsutil version" checking from gsutil.py
and just assume that if the file exists, then it's good, which
should shave about 1-2s off of each gsutil.py call.
Bug: 772740,776311
Change-Id: I4fef62cfd46a849afed1f095dd6a96069376d13d
Reviewed-on: https://chromium-review.googlesource.com/707758
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Run "gsutil" through "vpython". This ensures that the package set
accompanying it is completely controlled. This is notably important on
GCE instances, which "gsutil" expects to supply a
"google_compute_engine" package.
BUG=chromium:741001
TEST=local
- Ran `gsutil cp` on 64-bit Linux machine, seems to work.
- Ran `gsutil cp` on GCE instance, seems to work.
This is a reland of 23836a643f.
Change-Id: If87261abd505ef77f95b3c70b6bd23f3e798b41d
Reviewed-on: https://chromium-review.googlesource.com/656565
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
This reverts commit 23836a643f.
Reason for revert: This is breaking all the chromecast builders.
Original change's description:
> [gsutil] run through "vpython"
>
> Run "gsutil" through "vpython". This ensures that the package set
> accompanying it is completely controlled. This is notably important on
> GCE instances, which "gsutil" expects to supply a
> "google_compute_engine" package.
>
> BUG=None
> TEST=local
> - Ran `gsutil cp` on 64-bit Linux machine, seems to work.
> - Ran `gsutil cp` on GCE instance, seems to work.
>
> Change-Id: Id771bc4da4de22c941b0ace4031cb0f3364feeeb
> Reviewed-on: https://chromium-review.googlesource.com/527277
> Reviewed-by: Nodir Turakulov <nodir@chromium.org>
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Daniel Jacques <dnj@chromium.org>
TBR=iannucci@chromium.org,dnj@chromium.org,nodir@chromium.org
Change-Id: Ifc3d6389720abd0fb9e3fc83c0cdd5a63b5ff94d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: None
Reviewed-on: https://chromium-review.googlesource.com/656358
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Run "gsutil" through "vpython". This ensures that the package set
accompanying it is completely controlled. This is notably important on
GCE instances, which "gsutil" expects to supply a
"google_compute_engine" package.
BUG=None
TEST=local
- Ran `gsutil cp` on 64-bit Linux machine, seems to work.
- Ran `gsutil cp` on GCE instance, seems to work.
Change-Id: Id771bc4da4de22c941b0ace4031cb0f3364feeeb
Reviewed-on: https://chromium-review.googlesource.com/527277
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
In theory this shouldn't fire if gsutil is running without a tty (ie, in a subprocess).
In practice we're still seeing processes hang because it still thinks it's in a tty
This should in theory disable the prompt for good
BUG=577325,576979
Review URL: https://codereview.chromium.org/1587793002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@299714 0039d316-1c4b-4281-b951-d872f2087c98
- Update "gsutil.py" to be cooperatively safe when invoked
multiple times simultaneously.
- Allow the cache directory to be overridden by the
DEPOT_TOOLS_GSUTIL_BIN_DIR environment variable.
- Add a "--clean" flag to force "gsutil.py" to do a clean download.
BUG=chromium:452497
TEST=local
- for i in `seq 1 50`; do ./gsutil.py --clean -- version&; done
Review URL: https://codereview.chromium.org/1346213003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@296772 0039d316-1c4b-4281-b951-d872f2087c98
So that gsutil.py config works.
I would've preferred the execv solution, but apparently that didn't work on Windows :(
BUG=451551
Review URL: https://codereview.chromium.org/870093003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293790 0039d316-1c4b-4281-b951-d872f2087c98
Bruce Dawson
Aravind Vasudevan
William Hesse
Joanna Wang
Josip Sokcevic
Gavin Mak
Garrett Beaty
Takuto Ikuta
Dirk Pranke
Edward Lemur
Edward Lesmes
Quinten Yearsley
Raul Tambre
Caleb Rouleau
Ryan Tseng
Chris Nardi
Dan Jacques
Mike Bjorge
Dan Jacques
hinoka@chromium.org