* make RunTest's multiprocessing.Pool in the constructor of InputApi
to avoid getting tripped up by chdir manipulation.
* Don't do the split cyclic-import check when the invoker of the
Pylint presubmit checks explicitly sends cyclic import check
parameters via extra_args
* fix pseudobug where ownership of the files variable was unclear,
and pass all arguments on stdin (instead of mix of CLI + stdin).
* fix bug in pylint which caused it to manipulate sys.path before
spawning its subprocesses, which caused multiprocessing to fail
on windows.
* Note: This may carry a slight semantic change. Before, pylint would
add all .py files' directories to sys.path while checking any of
them. Now in parallel mode, pylint will only add the path of the
single file to sys.path. This behavior actually mirrors Python's
own behavior, so the check should be more-correct than before (and
should cut down on pylint import scanning time with very large
sys.path's).
* If someone encounters an issue with this, please note that the
GetPylint check also includes an extra_paths_list which is
expressly for this purpose.
R=dpranke@chromium.org, kbr@chromium.org, maruel@chromium.org
BUG=501012
Review URL: https://codereview.chromium.org/1208743002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295908 0039d316-1c4b-4281-b951-d872f2087c98
This allows to avoid conflicts with libraries in users' PATH
R=pgervais@chromium.org, tandrii@chromium.org
Review URL: https://codereview.chromium.org/1161803004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295497 0039d316-1c4b-4281-b951-d872f2087c98
Internal Rietveld exposes OAuth2 compatible API via /bots/* endpoints, not via
/* as public one does. See crbug.com/476690.
R=maruel@chromium.org
BUG=356813
Review URL: https://codereview.chromium.org/1082033002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294793 0039d316-1c4b-4281-b951-d872f2087c98
This CL introduces new top level command for managing cached auth tokens:
$ depot-tools-auth login codereview.chromium.org
$ depot-tools-auth info codereview.chromium.org
$ depot-tools-auth logout codereview.chromium.org
All scripts that use rietveld.Rietveld internally should be able to use cached
credentials created by 'depot-tools-auth' subcommand. Also 'depot-tools-auth'
is the only way to run login flow. If some scripts stumbles over expired or
revoked token, it dies with the error, asking user to run
'depot-tools-auth login <hostname>'.
Password login is still default. OAuth2 can be enabled by passing --oauth2 to
all scripts.
R=maruel@chromium.org
BUG=356813
Review URL: https://codereview.chromium.org/1074673002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294764 0039d316-1c4b-4281-b951-d872f2087c98
It is done in preparation for switching to OAuth2 as default (and only)
authentication method. Having all auth options handled by the same code makes it
easier to gradually add OAuth2 support.
As part of this, some options that would no longer work with OAuth2 (and that
are not being used from anywhere now, as far as I can tell) are removed:
* Passing account password for authentication via command line.
* Overriding 'Host' header when making requests to Rietveld (won't work with
SSL anyway).
* --account_type option (seems to be ClientLogin specific).
R=maruel@chromium.org
BUG=356813
Review URL: https://codereview.chromium.org/1075723002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294746 0039d316-1c4b-4281-b951-d872f2087c98
Use 'realpath' instead of 'abspath' for self-discovery. This fixes an
issue where 'gsutil' cannot find its dependencies when it is referenced
through a symbolic link.
Specifically, the symlink was: /b/scripts => /b/build/scripts
'gsutil' was called as:
/b/scripts/slave/../../../depot_tools/third_party/gsutil/gsutil
Because '/b/scripts/slave' resolves to '/b/build/scripts/slave', the three
parent directory operators actually resolve to '/b', but 'os.path.abspath'
blindly resolves them to '/'.
BUG=chromium:474442
TEST=live
- Signed onto failing bot, made this change, 'gsutil' works.
R=hinoka@chromium.org
Review URL: https://codereview.chromium.org/1060353002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294712 0039d316-1c4b-4281-b951-d872f2087c98
BUG=chromium:356813
TEST=local
- Deleted cookies file, ran without patch, 404.
- Deleted cookies file, ran on internal network, works!
Review URL: https://codereview.chromium.org/1068973002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294701 0039d316-1c4b-4281-b951-d872f2087c98
This is largely a bugfix release, so should be much easier to transition.
BUG=chromium:431514
TEST=ran on some code bases and checked output
Review URL: https://codereview.chromium.org/876793002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293806 0039d316-1c4b-4281-b951-d872f2087c98
This is largely a bugfix release, so should be much easier to transition.
BUG=chromium:431514
TEST=ran on some code bases and checked output
Review URL: https://codereview.chromium.org/753543006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293355 0039d316-1c4b-4281-b951-d872f2087c98
The reason cited for reverting was invalid, so reland the update.
BUG=chromium:431514
Review URL: https://codereview.chromium.org/739393004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293049 0039d316-1c4b-4281-b951-d872f2087c98
The current pylint is very old at this point. Pull in the latest version
as it supports a lot more features. Also need to fix the pylint wrapper
to actually update sys.path to use the local modules.
We include the six module since not all bots have it, and it's small.
This will trigger new warnings in files, but they look like legitimate
issues that should be fixed at some point.
BUG=chromium:431514
TEST=ran on some code bases and checked output
Review URL: https://codereview.chromium.org/741503002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@292988 0039d316-1c4b-4281-b951-d872f2087c98
This reverts commit 451939d582.
Throws an exception with a missing import, breaking Telemetry's
presubmit checks and presumably presubmit checks for all Python files
in Chromium. See comment on https://codereview.chromium.org/707353002/ .
BUG=chromium:431514
Review URL: https://codereview.chromium.org/719313003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@292955 0039d316-1c4b-4281-b951-d872f2087c98
The current pylint is very old at this point. Pull in the latest version
as it supports a lot more features. Also need to fix the pylint wrapper
to actually update sys.path to use the local modules.
This will trigger new warnings in files, but they look like legitimate
issues that should be fixed at some point.
BUG=chromium:431514
TEST=ran on some code bases and checked output
Review URL: https://codereview.chromium.org/707353002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@292954 0039d316-1c4b-4281-b951-d872f2087c98
This is already fixed upstream in gsutil.
BUG=None.
TEST=None.
Review URL: https://codereview.chromium.org/341833005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@280386 0039d316-1c4b-4281-b951-d872f2087c98
The "What is your project ID" has confused about every user of gsutil that I've
talked to, and isn't even useful in the context of download_from_google_storage
because its only good for "mb" on a bucket and "ls" without specifying a bucket,
neither of which this version of gsutil should ever do.
This CL:
* Removes the prompt for project ID
* Removes the 2 second delay which was meant for autolaunching a browser
* We don't want this behavior anyways because most of our devs have multiple
browser profiles open and they need to paste the URL into the correct profile.
* Removes support to autolaunch a broswer.
* Removes a lot of the extra messages. The prompt now looks like:
===
$ download_from_google_storage -g
Please navigate your browser to the following URL:
https://accounts.google.com/o/oauth2/auth?.....
Enter the authorization code: <code>
Boto config file "/usr/local/google/home/hinoka/.boto.depot_tools" created.
$
===
BUG=351015
Review URL: https://codereview.chromium.org/222973002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@267973 0039d316-1c4b-4281-b951-d872f2087c98
When rietveld.py overrides upload.py's logging module, it generally works fine...
until someone tries to run upload.py with any level of verbosity. Then the
calls to logging.getLogger and logging.INFO/logging.DEBUG fail. This patches
upload.py to properly use a module-scope logger so that rietveld.py doesn't
have to perform invasive surgery, and everything just works.
This version of upload.py taken from upstream Rietveld at:
changeset: 1267:d7b39eca7dbe
branch: chromium
R=djacques@chromium.org, maruel@chromium.org
Review URL: https://codereview.chromium.org/221423007
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@262793 0039d316-1c4b-4281-b951-d872f2087c98
Features:
- Non-verbose output is now limited to a one-line progress
indicator.
- Verbose output is now collated per subprocess. As soon as a
subprocess finishes, its full output is dumped to terminal.
- Verbose output is prefixed with timestamps representing elapsed
time since the beginning of the gclient invocation.
- git progress indicators ("Receiving objects", etc.) are limited to
one line every 10 seconds.
- In both verbose and non-verbose mode, if a failure occurs, the
full output of the failed update operation is dumped to terminal
just before exit.
- In the event that updates are progressing, but slowly,
"Still working" messages will be printed periodically, to pacify
users and buildbots.
BUG=
R=hinoka@google.com
Review URL: https://codereview.chromium.org/227163002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@262500 0039d316-1c4b-4281-b951-d872f2087c98
The Telemetry workflow happily uses this gsutil to download artifacts, but they would also like to use the same workflow to upload artifacts.
I don't see a big harm in enabling read_write access for prodaccess based gsutil since its already more restrictive than the normal gsutil auth flow. I could also be convinced to make this full_control, but that need has not arisen yet.
BUG= 341665
Review URL: https://codereview.chromium.org/148203012
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@259761 0039d316-1c4b-4281-b951-d872f2087c98
Reason for revert:
This causes download_from_google_storage to fail:
Failure: CA certificates file does not have expected SHA1 sum; expected: ed024a78d9327f8669b3b117d9eac9e3c9460e9b, actual: 433e5589520cd2f1f29dea3e077cfbc2b74fb78c.
You'll need to also update the expectations on the gsutil side, and then reland.
Test it by running "gsutil ls".
Original issue's description:
> Replaced boto certificate
>
> BUG=348233
>
> Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=259692
TBR=stip@chromium.org,rsleevi@chromium.org,pgervais@chromium.org
NOTREECHECKS=true
NOTRY=true
BUG=348233
Review URL: https://codereview.chromium.org/213963002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@259744 0039d316-1c4b-4281-b951-d872f2087c98
- Don't pylint files in .gitignore or .git/info/excludes
- Print full path to modules in pylint.
BUG=
Review URL: https://codereview.chromium.org/202773002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@257672 0039d316-1c4b-4281-b951-d872f2087c98
It looked like the bypass_prodaccess flag wasn't passed in all the way to
where it needed to be, causing it to be set to True by default at some point,
which caused the prodaccess path to never happen. This change takes the flag
and brings it all of the way.
BUG=337972
Review URL: https://codereview.chromium.org/147443002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@247049 0039d316-1c4b-4281-b951-d872f2087c98
By default, if loasd is not running, loas_check retries twice, waiting 5 seconds between retries. This is in case loasd is restarting due to reinstallation. Unfortunately, if the user has never run prodaccess, this check takes an extra 10 seconds. So, just disable the retries.
BUG=
Review URL: https://codereview.chromium.org/137853024
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@245399 0039d316-1c4b-4281-b951-d872f2087c98
Code path:
1. plugins.sso_auth is imported, which adds the AuthHandler class to the global state.
2. HasConfiguredCredentials() in gslib/utils.py is called by gsutil, and will return true if "prodaccess" exists on the system, which tells the system that we don't want a no-op auth handler.
3. When a command is called, all the auth handlers are cycled through and sso_auth.SSOAuth is called, which calls a stubby command to emit a gaiamint'ed oauth2 access token, which is then used as the Authorization Header
if --bypass_prodaccess is passed in, then:
1. HasConfiguredCredentials() will bypass the check for prodaccess, as if it didn't exist.
2. plugins.sso_auth does not get imported.
Which will essentially cause gsutil to behave as if this patch never existed.
So the expected behavior is:
=.boto file does not exist, prodaccess exists, but unauthenticated=
Failure: No handler was ready to authenticate. 3 handlers were checked. ['OAuth2Auth', 'HmacAuthV1Handler', 'SSOAuth'] Check your credentials.
=.boto file exists, prodaccess exists, but unauthenticated=
sso_auth will raise NotReadyToAuthenticate, and the .boto file will be used instead
=.boto file exists, prodaccess exists, authenticated=
sso_auth will be run _after_ the default gsutil authenticator, which causes the sso_auth to be used over whatever the default authentication is.
bypass_prodaccess is passed in by default to upload_to_google_storage because we expect people who use upload_to_google_storage to not need prodaccess and have their own boto file already. Also the sso_auth plugin will only request a readonlyi token, which will not work for uploading.
BUG=258152
Review URL: https://codereview.chromium.org/86123002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@240266 0039d316-1c4b-4281-b951-d872f2087c98
iannucci@chromium.org
sergiyb@chromium.org
rmistry@google.com
sheyang@chromium.org
akuegel@chromium.org
sheyang@google.com
vadimsh@chromium.org
dnj@chromium.org
dnj@google.com
vapier@chromium.org
smut@google.com
jochen@chromium.org
kbr@chromium.org
jrobbins@chromium.org
jrobbins@chromium.org
dtu@chromium.org
sheyang@chromium.org
hinoka@chromium.org
agable@chromium.org
szager@chromium.org
pgervais@chromium.org
hinoka@google.com