git cl upload asks the user to provide a patchset title.
If the user makes a commit per patchset, the subject of the commit may
be exactly what patchset title should be.
R=dnj@chromium.org, vadimsh@chromium.org
BUG=582419
Review-Url: https://codereview.chromium.org/1868313002
If the stderr buffer used in git diff became full
before stdout was completely consumed, git cl upload
would deadlock.
This also sets GIT_PAGER to be perfectly sure no pager
is triggered by the git commands run. I haven't
seen that as a problem but it's a reasonable (and
common) precaution.
BUG=558726
Review URL: https://codereview.chromium.org/1544813002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@298090 0039d316-1c4b-4281-b951-d872f2087c98
Reason for revert:
This CL over-aggressively removed the Windows bootstrap code used by /gclient.bat, which in turn downloads/installs the Windows toolchain, including Python. Labs can no longer provision Windows bots because of this.
update_depot_tools.bat did more than initialize the virtualenv. It also downloaded and installed Python, Git, and Subversion on Windows systems.
I'm reverting to restore the toolchain so we can create new Windows bots.
Original issue's description:
> Removed virtualenv from depot_tools
>
> This effectively reverts http://crrev.com/1195423002 and
> http://crrev.com/1205873002.
>
> R=pgervais@chromium.org, tandrii@chromium.org
> TBR=pgervais@chromium.org # i wanna get my Fixit credit today :-)
> BUG=542922,503067
>
> Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=297491
TBR=pgervais@chromium.org,tandrii@chromium.org,sergiyb@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=542922,503067
Review URL: https://codereview.chromium.org/1431173002 .
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@297493 0039d316-1c4b-4281-b951-d872f2087c98
* make RunTest's multiprocessing.Pool in the constructor of InputApi
to avoid getting tripped up by chdir manipulation.
* Don't do the split cyclic-import check when the invoker of the
Pylint presubmit checks explicitly sends cyclic import check
parameters via extra_args
* fix pseudobug where ownership of the files variable was unclear,
and pass all arguments on stdin (instead of mix of CLI + stdin).
* fix bug in pylint which caused it to manipulate sys.path before
spawning its subprocesses, which caused multiprocessing to fail
on windows.
* Note: This may carry a slight semantic change. Before, pylint would
add all .py files' directories to sys.path while checking any of
them. Now in parallel mode, pylint will only add the path of the
single file to sys.path. This behavior actually mirrors Python's
own behavior, so the check should be more-correct than before (and
should cut down on pylint import scanning time with very large
sys.path's).
* If someone encounters an issue with this, please note that the
GetPylint check also includes an extra_paths_list which is
expressly for this purpose.
R=dpranke@chromium.org, kbr@chromium.org, maruel@chromium.org
BUG=501012
Review URL: https://codereview.chromium.org/1208743002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295908 0039d316-1c4b-4281-b951-d872f2087c98
This allows to avoid conflicts with libraries in users' PATH
R=pgervais@chromium.org, tandrii@chromium.org
Review URL: https://codereview.chromium.org/1161803004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295497 0039d316-1c4b-4281-b951-d872f2087c98
Internal Rietveld exposes OAuth2 compatible API via /bots/* endpoints, not via
/* as public one does. See crbug.com/476690.
R=maruel@chromium.org
BUG=356813
Review URL: https://codereview.chromium.org/1082033002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294793 0039d316-1c4b-4281-b951-d872f2087c98
This CL introduces new top level command for managing cached auth tokens:
$ depot-tools-auth login codereview.chromium.org
$ depot-tools-auth info codereview.chromium.org
$ depot-tools-auth logout codereview.chromium.org
All scripts that use rietveld.Rietveld internally should be able to use cached
credentials created by 'depot-tools-auth' subcommand. Also 'depot-tools-auth'
is the only way to run login flow. If some scripts stumbles over expired or
revoked token, it dies with the error, asking user to run
'depot-tools-auth login <hostname>'.
Password login is still default. OAuth2 can be enabled by passing --oauth2 to
all scripts.
R=maruel@chromium.org
BUG=356813
Review URL: https://codereview.chromium.org/1074673002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294764 0039d316-1c4b-4281-b951-d872f2087c98
It is done in preparation for switching to OAuth2 as default (and only)
authentication method. Having all auth options handled by the same code makes it
easier to gradually add OAuth2 support.
As part of this, some options that would no longer work with OAuth2 (and that
are not being used from anywhere now, as far as I can tell) are removed:
* Passing account password for authentication via command line.
* Overriding 'Host' header when making requests to Rietveld (won't work with
SSL anyway).
* --account_type option (seems to be ClientLogin specific).
R=maruel@chromium.org
BUG=356813
Review URL: https://codereview.chromium.org/1075723002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294746 0039d316-1c4b-4281-b951-d872f2087c98
Use 'realpath' instead of 'abspath' for self-discovery. This fixes an
issue where 'gsutil' cannot find its dependencies when it is referenced
through a symbolic link.
Specifically, the symlink was: /b/scripts => /b/build/scripts
'gsutil' was called as:
/b/scripts/slave/../../../depot_tools/third_party/gsutil/gsutil
Because '/b/scripts/slave' resolves to '/b/build/scripts/slave', the three
parent directory operators actually resolve to '/b', but 'os.path.abspath'
blindly resolves them to '/'.
BUG=chromium:474442
TEST=live
- Signed onto failing bot, made this change, 'gsutil' works.
R=hinoka@chromium.org
Review URL: https://codereview.chromium.org/1060353002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294712 0039d316-1c4b-4281-b951-d872f2087c98
BUG=chromium:356813
TEST=local
- Deleted cookies file, ran without patch, 404.
- Deleted cookies file, ran on internal network, works!
Review URL: https://codereview.chromium.org/1068973002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294701 0039d316-1c4b-4281-b951-d872f2087c98
This is largely a bugfix release, so should be much easier to transition.
BUG=chromium:431514
TEST=ran on some code bases and checked output
Review URL: https://codereview.chromium.org/876793002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293806 0039d316-1c4b-4281-b951-d872f2087c98
This is largely a bugfix release, so should be much easier to transition.
BUG=chromium:431514
TEST=ran on some code bases and checked output
Review URL: https://codereview.chromium.org/753543006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293355 0039d316-1c4b-4281-b951-d872f2087c98
The reason cited for reverting was invalid, so reland the update.
BUG=chromium:431514
Review URL: https://codereview.chromium.org/739393004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293049 0039d316-1c4b-4281-b951-d872f2087c98
The current pylint is very old at this point. Pull in the latest version
as it supports a lot more features. Also need to fix the pylint wrapper
to actually update sys.path to use the local modules.
We include the six module since not all bots have it, and it's small.
This will trigger new warnings in files, but they look like legitimate
issues that should be fixed at some point.
BUG=chromium:431514
TEST=ran on some code bases and checked output
Review URL: https://codereview.chromium.org/741503002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@292988 0039d316-1c4b-4281-b951-d872f2087c98
This reverts commit 451939d582.
Throws an exception with a missing import, breaking Telemetry's
presubmit checks and presumably presubmit checks for all Python files
in Chromium. See comment on https://codereview.chromium.org/707353002/ .
BUG=chromium:431514
Review URL: https://codereview.chromium.org/719313003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@292955 0039d316-1c4b-4281-b951-d872f2087c98
The current pylint is very old at this point. Pull in the latest version
as it supports a lot more features. Also need to fix the pylint wrapper
to actually update sys.path to use the local modules.
This will trigger new warnings in files, but they look like legitimate
issues that should be fixed at some point.
BUG=chromium:431514
TEST=ran on some code bases and checked output
Review URL: https://codereview.chromium.org/707353002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@292954 0039d316-1c4b-4281-b951-d872f2087c98
This is already fixed upstream in gsutil.
BUG=None.
TEST=None.
Review URL: https://codereview.chromium.org/341833005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@280386 0039d316-1c4b-4281-b951-d872f2087c98
The "What is your project ID" has confused about every user of gsutil that I've
talked to, and isn't even useful in the context of download_from_google_storage
because its only good for "mb" on a bucket and "ls" without specifying a bucket,
neither of which this version of gsutil should ever do.
This CL:
* Removes the prompt for project ID
* Removes the 2 second delay which was meant for autolaunching a browser
* We don't want this behavior anyways because most of our devs have multiple
browser profiles open and they need to paste the URL into the correct profile.
* Removes support to autolaunch a broswer.
* Removes a lot of the extra messages. The prompt now looks like:
===
$ download_from_google_storage -g
Please navigate your browser to the following URL:
https://accounts.google.com/o/oauth2/auth?.....
Enter the authorization code: <code>
Boto config file "/usr/local/google/home/hinoka/.boto.depot_tools" created.
$
===
BUG=351015
Review URL: https://codereview.chromium.org/222973002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@267973 0039d316-1c4b-4281-b951-d872f2087c98
When rietveld.py overrides upload.py's logging module, it generally works fine...
until someone tries to run upload.py with any level of verbosity. Then the
calls to logging.getLogger and logging.INFO/logging.DEBUG fail. This patches
upload.py to properly use a module-scope logger so that rietveld.py doesn't
have to perform invasive surgery, and everything just works.
This version of upload.py taken from upstream Rietveld at:
changeset: 1267:d7b39eca7dbe
branch: chromium
R=djacques@chromium.org, maruel@chromium.org
Review URL: https://codereview.chromium.org/221423007
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@262793 0039d316-1c4b-4281-b951-d872f2087c98
Features:
- Non-verbose output is now limited to a one-line progress
indicator.
- Verbose output is now collated per subprocess. As soon as a
subprocess finishes, its full output is dumped to terminal.
- Verbose output is prefixed with timestamps representing elapsed
time since the beginning of the gclient invocation.
- git progress indicators ("Receiving objects", etc.) are limited to
one line every 10 seconds.
- In both verbose and non-verbose mode, if a failure occurs, the
full output of the failed update operation is dumped to terminal
just before exit.
- In the event that updates are progressing, but slowly,
"Still working" messages will be printed periodically, to pacify
users and buildbots.
BUG=
R=hinoka@google.com
Review URL: https://codereview.chromium.org/227163002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@262500 0039d316-1c4b-4281-b951-d872f2087c98
The Telemetry workflow happily uses this gsutil to download artifacts, but they would also like to use the same workflow to upload artifacts.
I don't see a big harm in enabling read_write access for prodaccess based gsutil since its already more restrictive than the normal gsutil auth flow. I could also be convinced to make this full_control, but that need has not arisen yet.
BUG= 341665
Review URL: https://codereview.chromium.org/148203012
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@259761 0039d316-1c4b-4281-b951-d872f2087c98
Reason for revert:
This causes download_from_google_storage to fail:
Failure: CA certificates file does not have expected SHA1 sum; expected: ed024a78d9327f8669b3b117d9eac9e3c9460e9b, actual: 433e5589520cd2f1f29dea3e077cfbc2b74fb78c.
You'll need to also update the expectations on the gsutil side, and then reland.
Test it by running "gsutil ls".
Original issue's description:
> Replaced boto certificate
>
> BUG=348233
>
> Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=259692
TBR=stip@chromium.org,rsleevi@chromium.org,pgervais@chromium.org
NOTREECHECKS=true
NOTRY=true
BUG=348233
Review URL: https://codereview.chromium.org/213963002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@259744 0039d316-1c4b-4281-b951-d872f2087c98
- Don't pylint files in .gitignore or .git/info/excludes
- Print full path to modules in pylint.
BUG=
Review URL: https://codereview.chromium.org/202773002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@257672 0039d316-1c4b-4281-b951-d872f2087c98
It looked like the bypass_prodaccess flag wasn't passed in all the way to
where it needed to be, causing it to be set to True by default at some point,
which caused the prodaccess path to never happen. This change takes the flag
and brings it all of the way.
BUG=337972
Review URL: https://codereview.chromium.org/147443002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@247049 0039d316-1c4b-4281-b951-d872f2087c98
By default, if loasd is not running, loas_check retries twice, waiting 5 seconds between retries. This is in case loasd is restarting due to reinstallation. Unfortunately, if the user has never run prodaccess, this check takes an extra 10 seconds. So, just disable the retries.
BUG=
Review URL: https://codereview.chromium.org/137853024
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@245399 0039d316-1c4b-4281-b951-d872f2087c98
Code path:
1. plugins.sso_auth is imported, which adds the AuthHandler class to the global state.
2. HasConfiguredCredentials() in gslib/utils.py is called by gsutil, and will return true if "prodaccess" exists on the system, which tells the system that we don't want a no-op auth handler.
3. When a command is called, all the auth handlers are cycled through and sso_auth.SSOAuth is called, which calls a stubby command to emit a gaiamint'ed oauth2 access token, which is then used as the Authorization Header
if --bypass_prodaccess is passed in, then:
1. HasConfiguredCredentials() will bypass the check for prodaccess, as if it didn't exist.
2. plugins.sso_auth does not get imported.
Which will essentially cause gsutil to behave as if this patch never existed.
So the expected behavior is:
=.boto file does not exist, prodaccess exists, but unauthenticated=
Failure: No handler was ready to authenticate. 3 handlers were checked. ['OAuth2Auth', 'HmacAuthV1Handler', 'SSOAuth'] Check your credentials.
=.boto file exists, prodaccess exists, but unauthenticated=
sso_auth will raise NotReadyToAuthenticate, and the .boto file will be used instead
=.boto file exists, prodaccess exists, authenticated=
sso_auth will be run _after_ the default gsutil authenticator, which causes the sso_auth to be used over whatever the default authentication is.
bypass_prodaccess is passed in by default to upload_to_google_storage because we expect people who use upload_to_google_storage to not need prodaccess and have their own boto file already. Also the sso_auth plugin will only request a readonlyi token, which will not work for uploading.
BUG=258152
Review URL: https://codereview.chromium.org/86123002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@240266 0039d316-1c4b-4281-b951-d872f2087c98
boto config does the right thing by using the http_proxy env var to set the proxy,
and gsutil removes that var which isn't the behavior we want.
This should allow gsutil to respect the http_proxy env var.
BUG=318478
Review URL: https://codereview.chromium.org/68763013
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@235818 0039d316-1c4b-4281-b951-d872f2087c98
This fixes license headers.
Interestingly, the pylint workaround at r133705 doesn't seem to be necessary
anymore.
R=stip@chromium.org
BUG=
Review URL: https://codereview.chromium.org/54603003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@232152 0039d316-1c4b-4281-b951-d872f2087c98
Add an option to apply_issue to fail if anonymous access
triggers a login prompt.
- cherry-pick upload.py rietveld cl 37c73ece82d0 which
allows clients to request no authentication.
BUG=240634
Review URL: https://chromiumcodereview.appspot.com/20634003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@214023 0039d316-1c4b-4281-b951-d872f2087c98
This pulls r1084 from the chromium on upstream rietveld.
R=jparent@chromium.org, maruel@chromium.org
BUG=247791
Review URL: https://chromiumcodereview.appspot.com/16744006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@206743 0039d316-1c4b-4281-b951-d872f2087c98
Make upload.py treat images as binaries, so they upload correctly and display in Reitveld.
BUG=227346
Review URL: https://chromiumcodereview.appspot.com/14820015
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@198833 0039d316-1c4b-4281-b951-d872f2087c98
First, the environment variable for the child process was created but not
specified to subprocess.call().
Second, third_party/logilab/__init__.py tried to initialize itself with
pkg_resources.
TBR=chrisha@chromium.org
BUG=
TEST=
Review URL: https://chromiumcodereview.appspot.com/10582031
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@143111 0039d316-1c4b-4281-b951-d872f2087c98
This CL adds pylint (and its dependencies) to third_party. It plumbs them into presubmit_canned_checks, and exposes a command-line tool to access pylint.
BUG=
TEST=
Review URL: https://chromiumcodereview.appspot.com/10447014
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@143016 0039d316-1c4b-4281-b951-d872f2087c98
BUG=None
TEST=gcl upload now uploads any type of binary file, not just images.
Review URL: https://chromiumcodereview.appspot.com/10332313
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@138469 0039d316-1c4b-4281-b951-d872f2087c98
Pylint fails to parse generated members in a __init__() function.
R=cmp@chromium.org
BUG=
TEST=Manually tested on linux, cygwin and windows
Review URL: http://codereview.chromium.org/10202010
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@133705 0039d316-1c4b-4281-b951-d872f2087c98
This version of upload.py is slightly less annoying, it won't cancel out if no
patchset title is provided and disable colors all the time.
R=cmp@chromium.org
BUG=
TEST=
Review URL: http://codereview.chromium.org/9233057
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@121826 0039d316-1c4b-4281-b951-d872f2087c98
It also removes the need of manually creating a subject argument.
Other related changes in this CL:
- Reenable the prompt for patchset title in gcl. I'm not sure why it was disabled.
- Remove git cl upload --desc_from_logs flag. --force is already meaningful.
R=cmp@chromium.org
BUG=
TEST=
Review URL: http://codereview.chromium.org/9193023
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@119066 0039d316-1c4b-4281-b951-d872f2087c98
This CL adds the egg distribution of setuptools to third_party. A follow-up CL will use this to install various required third-party eggs in a depot_tools/site-packages directory.
TBR=maruel@chromium.org
BUG=
TEST=
Review URL: http://codereview.chromium.org/8953018
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@114635 0039d316-1c4b-4281-b951-d872f2087c98
The main fix is 'Accept new empty files, like __init__.py'
TBR=dpranke@chromium.org
BUG=
TEST=
Review URL: http://codereview.chromium.org/8519005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@109431 0039d316-1c4b-4281-b951-d872f2087c98
vapier
nodir
tandrii@chromium.org
bratell@opera.com
sergiyb@chromium.org
akuegel@chromium.org
machenbach@chromium.org
dnj@google.com
stip@chromium.org
petermayo@chromium.org
iannucci@chromium.org
rmistry@google.com
sheyang@chromium.org
sheyang@google.com
vadimsh@chromium.org
dnj@chromium.org
smut@google.com
jochen@chromium.org
kbr@chromium.org
jrobbins@chromium.org
jrobbins@chromium.org
dtu@chromium.org
sheyang@chromium.org
hinoka@chromium.org
agable@chromium.org
szager@chromium.org
pgervais@chromium.org
hinoka@google.com
iannucci@chromium.org
maruel@chromium.org
ilevy@chromium.org
jparent@chromium.org
chrisha@chromium.org
chrisha@google.com
rogerta@chromium.org
jam@chromium.org