I fixed the overlapping argument and fixed up the tests so that the actual sha1 hash is now passed (since the tool actually check now that the downloaded file hash matches)
BUG=
R=hinoka@google.com
Review URL: https://codereview.chromium.org/1285423002 .
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@296346 0039d316-1c4b-4281-b951-d872f2087c98
This does one last check to see if the file downloaded by
download_from_google_storage.py actually matches its sha1
BUG=
Review URL: https://codereview.chromium.org/1252313005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@296155 0039d316-1c4b-4281-b951-d872f2087c98
Reason for revert:
This conflicts with (and duplicates?) the -z option below and has broken one of the chrome android downstream bots.
Safer to revert than trying to fix
Original issue's description:
> Add support for tar.gz archive files to download from download_from_google_storage
>
> Also, add support for directly setting the public read flag on uploaded files.
>
> The support for tar.gz allows the uploaded bundle to be a single tar.gz file instead of a lot of individual files using the directory support already in. The benefit here is that it is much easier to update the dependency. Simply clean out the existing files, copy in the new ones, create a tar.gz file, with the same name as the directory + 'tar.gz'. If the directory name and file name does not match up we will not clean up existing artifacts on download (i.e., there can be left over files after extracting).
>
> I am doing this because I am moving a bunch of the dart dependencies to gcs, and a lot of our dependencies is much easier to manage with this in. If you don't like this, I can simply wrap the download script in another python script and do the logic there, but this may be handy for other people as well.
>
> Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=295872
TBR=hinoka@google.com,hinoka@chromium.org,azarchs@chromium.org,ricow@google.com
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
Review URL: https://codereview.chromium.org/1209033006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295883 0039d316-1c4b-4281-b951-d872f2087c98
Also, add support for directly setting the public read flag on uploaded files.
The support for tar.gz allows the uploaded bundle to be a single tar.gz file instead of a lot of individual files using the directory support already in. The benefit here is that it is much easier to update the dependency. Simply clean out the existing files, copy in the new ones, create a tar.gz file, with the same name as the directory + 'tar.gz'. If the directory name and file name does not match up we will not clean up existing artifacts on download (i.e., there can be left over files after extracting).
I am doing this because I am moving a bunch of the dart dependencies to gcs, and a lot of our dependencies is much easier to manage with this in. If you don't like this, I can simply wrap the download script in another python script and do the logic there, but this may be handy for other people as well.
Review URL: https://codereview.chromium.org/807463005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295872 0039d316-1c4b-4281-b951-d872f2087c98
This does not seem to be run by the bots, otherwise they should be red
R=hinoka@google.com, iannucci@google.com
BUG=
Review URL: https://codereview.chromium.org/1188643008
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295741 0039d316-1c4b-4281-b951-d872f2087c98
A very common pattern for the users of this script is to have binaries in
three different folders, eg:
some/folder/win/bin.exe
some/folder/mac/bin
some/folder/linux/bin
Instead of using --platform to specify three different paths, we can recognize
this usage and pass in --auto_platform, and then --directory some/folder/ as the
target.
When enumerating files, it will match each individual file to see if they have
(win|mac|linux) as a parent folder name, and filter out non-matching platforms.
BUG=
TEST= src/third_party/clang_format/bin$ download_from_google_storage --auto_platform --no_auth -b chromium-clang-fomat -dr .
0> Downloading ./linux/clang-format...
Review URL: https://codereview.chromium.org/126893002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@245643 0039d316-1c4b-4281-b951-d872f2087c98
Code path:
1. plugins.sso_auth is imported, which adds the AuthHandler class to the global state.
2. HasConfiguredCredentials() in gslib/utils.py is called by gsutil, and will return true if "prodaccess" exists on the system, which tells the system that we don't want a no-op auth handler.
3. When a command is called, all the auth handlers are cycled through and sso_auth.SSOAuth is called, which calls a stubby command to emit a gaiamint'ed oauth2 access token, which is then used as the Authorization Header
if --bypass_prodaccess is passed in, then:
1. HasConfiguredCredentials() will bypass the check for prodaccess, as if it didn't exist.
2. plugins.sso_auth does not get imported.
Which will essentially cause gsutil to behave as if this patch never existed.
So the expected behavior is:
=.boto file does not exist, prodaccess exists, but unauthenticated=
Failure: No handler was ready to authenticate. 3 handlers were checked. ['OAuth2Auth', 'HmacAuthV1Handler', 'SSOAuth'] Check your credentials.
=.boto file exists, prodaccess exists, but unauthenticated=
sso_auth will raise NotReadyToAuthenticate, and the .boto file will be used instead
=.boto file exists, prodaccess exists, authenticated=
sso_auth will be run _after_ the default gsutil authenticator, which causes the sso_auth to be used over whatever the default authentication is.
bypass_prodaccess is passed in by default to upload_to_google_storage because we expect people who use upload_to_google_storage to not need prodaccess and have their own boto file already. Also the sso_auth plugin will only request a readonlyi token, which will not work for uploading.
BUG=258152
Review URL: https://codereview.chromium.org/86123002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@240266 0039d316-1c4b-4281-b951-d872f2087c98
It appears to be parsed and ignored, seems like a bug to me.
BUG=None
TEST=local/manual.
Review URL: https://chromiumcodereview.appspot.com/17265012
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@207404 0039d316-1c4b-4281-b951-d872f2087c98
When a developer runs download_from_google_storage, and they don't have a .boto
file, the tool automatically runs "gsutil config" to create one for them.
Unfortunately, a side consequence is that if a bot runs the script, and it has
a boto file that 403's, then it would run "gsutil config" which moves the .boto
to .boto.bak, and creates an empty .boto file. This should not be the intended
action.
This CL changes so that "gsutil config" is not called, but instead just fails
with a message telling the dev to run that command.
TBR=maruel@chromium.org
Review URL: https://codereview.chromium.org/16072023
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@203824 0039d316-1c4b-4281-b951-d872f2087c98
continuation of: https://codereview.chromium.org/11664024
Moved it from chrome/trunk/src/build to depot_tools/
BUG=153360
TEST=two unittests included in tests/
For end-to-end testing, check out a large directory. Run
find . -name .svn -prune -o -size +1000k -type f -print0 | upload_to_google_storage.py -b chrome-artifacts -0 -
(replacing chrome-artifacts with an upload-able bucket)
to test upload
run "find . -name .svn -prune -o -size +1000k -type f -print0 | xargs -0 rm" to remove the files uploaded. Check that the large binary files have been removed
run "download_from_google_storage.py -r -d -b chrome-artifacts ." to download the files again.
Review URL: https://chromiumcodereview.appspot.com/12042069
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@187951 0039d316-1c4b-4281-b951-d872f2087c98
ricow@chromium.org
hinoka@chromium.org
boliu@chromium.org
ricow@google.com
hinoka@google.com
hinoka@chromium.org
scottmg@chromium.org
petermayo@chromium.org