If os.kill(os.getpid(), signal.CTRL_C_EVENT) doesn't do what the
test author hopes it will do, the test will hang until it has calculated
a million out-of-thread/process results. That is approximately an
infinite hang.
BUG=
Review URL: https://codereview.chromium.org/110883004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@240319 0039d316-1c4b-4281-b951-d872f2087c98
Code path:
1. plugins.sso_auth is imported, which adds the AuthHandler class to the global state.
2. HasConfiguredCredentials() in gslib/utils.py is called by gsutil, and will return true if "prodaccess" exists on the system, which tells the system that we don't want a no-op auth handler.
3. When a command is called, all the auth handlers are cycled through and sso_auth.SSOAuth is called, which calls a stubby command to emit a gaiamint'ed oauth2 access token, which is then used as the Authorization Header
if --bypass_prodaccess is passed in, then:
1. HasConfiguredCredentials() will bypass the check for prodaccess, as if it didn't exist.
2. plugins.sso_auth does not get imported.
Which will essentially cause gsutil to behave as if this patch never existed.
So the expected behavior is:
=.boto file does not exist, prodaccess exists, but unauthenticated=
Failure: No handler was ready to authenticate. 3 handlers were checked. ['OAuth2Auth', 'HmacAuthV1Handler', 'SSOAuth'] Check your credentials.
=.boto file exists, prodaccess exists, but unauthenticated=
sso_auth will raise NotReadyToAuthenticate, and the .boto file will be used instead
=.boto file exists, prodaccess exists, authenticated=
sso_auth will be run _after_ the default gsutil authenticator, which causes the sso_auth to be used over whatever the default authentication is.
bypass_prodaccess is passed in by default to upload_to_google_storage because we expect people who use upload_to_google_storage to not need prodaccess and have their own boto file already. Also the sso_auth plugin will only request a readonlyi token, which will not work for uploading.
BUG=258152
Review URL: https://codereview.chromium.org/86123002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@240266 0039d316-1c4b-4281-b951-d872f2087c98
This is useful to diagnose slow tests. Make this multiprocessing aware. Stop
printing when the commands are added but use the proper message system instead.
R=iannucci@chromium.org
BUG=
Review URL: https://codereview.chromium.org/99453003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@238388 0039d316-1c4b-4281-b951-d872f2087c98
I've unified the checks for language specific exceptions to line length. There is a behavior change where cpp compiler directives no longer have a maximum line length (previously they were restricted to 150% of max line length). This change was made to match the behavior for java files.
BUG=NONE
Review URL: https://codereview.chromium.org/100163002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@238268 0039d316-1c4b-4281-b951-d872f2087c98
Tell download tools/hooks/commit-msg mannually if it looks broken.
chrome-internal-reviews.googlesource.com requires
authentication to access tools/hooks/commit-msg, and
we'll get Google Sign-in page rather than expected shell script.
R=szager@chromium.org,maruel@chromium.org,ilevy@chromium.org
BUG=
Review URL: https://codereview.chromium.org/87173002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@237454 0039d316-1c4b-4281-b951-d872f2087c98
This is a serious bug, if a version is unpinned, it would become stuck at
whatever a local branch 'master' may point to. This is extremely bad when a
pinned dependency is unpinned on purpose, for example on a canary master. We
reproduced this problem on the chromium.swarm master by setting a custom_vars
swarming_revision to '', which then stopped syncing at the expected
origin/master commit but instead got stuck to what happened to be a local master
branch.
This happens on git dependencies if the main solution is using svn. This doesn't
happen on git checkouts using .DEPS.git due to bug 323233.
The issue looks like:
________ running 'git reset --hard HEAD' in '/path/src/tools/swarming_client'
HEAD is now at 4727bd5 Some commit
Checked out revision d908a546e28d1e9f85f5690cf6c3a080f06ba711
Note that HEAD and what is checked out do not match.
Also reduce the hardcoding of 'origin' by creating a variable named 'remote'.
R=petermayo@chromium.org
BUG=322961
Review URL: https://codereview.chromium.org/85473007
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@237332 0039d316-1c4b-4281-b951-d872f2087c98
Compatible with any git topology (multiple roots, weird branching/merging, etc.)
I can't get it to be any faster (in python). Suggestions welcome :).
On z600/linux, this takes 5.1s to calculate the initial count for 2e3de954ef0a
(HEAD on src.git at the time of writing). Subsequent lookups take ~0.06s. For
reference, this machine takes 3s to just list the revisions in sorted order
without any additional processing (using rev-list).
All calculations are stored in a git-notes-style ref with the exception that the
leaf 'tree' object which would normally be stored in a git-notes world is
replaced with a packed binary file which consists of records [hash int]. Each
run of this script will create only 1 commit object on this internal ref which
will have as its parents:
* The previous git number commit
* All of the target commits we calculated numbers for.
This ref is then excluded on subsequent invocations of rev-list, which means that
git-number will only ever process commit objects which it hasn't already
calculated a value for. It also prevents you from attempting to number this
special ref :).
This implementation only has a 1-byte fanout which seems to be the best
performance for the repos we're dealing with (i.e. on the order of 500k commit
objects). Bumping this up to a 2-byte fanout became extremely slow (I suspect
the internal caching structures I'm using are not efficient in this mode and
could be improved). Using no fanout is slower than the 1 byte fanout for lookups
by about 30%.
R=agable@chromium.org, stip@chromium.org, szager@chromium.org
BUG=280154,309692,skia:1639
Review URL: https://codereview.chromium.org/26109002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@236035 0039d316-1c4b-4281-b951-d872f2087c98
Previously this relied on being able to track branches which didn't have a
remote refspec. Now we add a fake origin for the git-svn test repo, and use
--prefix so that git has a remote to track and the refs are correctly
namespaced.
Should also be backwards compatible with older versions of git.
R=maruel@chromium.org
BUG=
Review URL: https://codereview.chromium.org/61893006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@233649 0039d316-1c4b-4281-b951-d872f2087c98
This is needed for a further CL to unify TS and CQ using PRESUBMIT.py.
BUG=278554
Review URL: https://codereview.chromium.org/54373011
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@232813 0039d316-1c4b-4281-b951-d872f2087c98
Add retry for all git operations that speak to a remote. This
should smooth out issues with the git/gerrit-on-borg service.
Review URL: https://codereview.chromium.org/26234004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@229219 0039d316-1c4b-4281-b951-d872f2087c98
These are run for a given dependency after it has been synced but before its
DEPS have been synced. This will help to switch Chromium to depend on Skia's
git repository (skia:1638).
Review URL: https://codereview.chromium.org/25322002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@228651 0039d316-1c4b-4281-b951-d872f2087c98
The motivation for this CL is:
When a project is turned on to use GitCheckout with try job verifiers, then the CQ crashes because of line 64 in http://src.chromium.org/viewvc/chrome/trunk/tools/commit-queue/pending_manager.py
This is because revision can only be one of (None, int, unicode). This crash only happens when try jobs are used (which is why I did not see it before).
This CL changes the return type to be unicode instead of string.
Review URL: https://chromiumcodereview.appspot.com/25686005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@226743 0039d316-1c4b-4281-b951-d872f2087c98
This prevents private CLs to be automatically CC-ed to public mailing lists
unintentionally.
Review URL: https://chromiumcodereview.appspot.com/24257014
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@225925 0039d316-1c4b-4281-b951-d872f2087c98
This is in order to support 'features' like got_revision for build systems
unwilling to scrape the human-readable output of gclient in order to extract
basic information :)
R=agable@chromium.org, maruel@chromium.org
BUG=265618
Review URL: https://chromiumcodereview.appspot.com/23753008
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@225046 0039d316-1c4b-4281-b951-d872f2087c98
This solves a problem with "os.rename" sometimes failing with an
exception after cloning a dependency to a temporary directory. It's
possible the dying git processes still hold a handle to the directory.
Since gclient delete the temporary directory regardless of the success
of the process, it results in a lot of GB downloaded for nothing.
SafeRename solves this by retrying a few times if the renaming fails,
sleeping one second every time to get other processes the time to
release their lock on the directory. It gives up retrying after 15 times.
BUG=
R=maruel@chromium.org
Review URL: https://chromiumcodereview.appspot.com/23875041
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@224372 0039d316-1c4b-4281-b951-d872f2087c98
Previously we would suggest owners for the whole CL, rather than just
the files currently missing owner approval.
R=maruel@chromium.org
BUG=130978
Review URL: https://chromiumcodereview.appspot.com/23591064
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@224183 0039d316-1c4b-4281-b951-d872f2087c98
Update subcommand to support examples, always disable format_description and
format_epilog and add colors when enabled.
R=iannucci@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/22824018
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@218180 0039d316-1c4b-4281-b951-d872f2087c98
Currently the output is:
Found line ending with white spaces in:
cc/trees/layer_tree_host_impl_unittest.cc, line 379
This format is extremely annoying because you must combine the
file and line number together by hand. A more standard format
for expressing a position in a file is filename:lineno. This
changes the format to match:
Found line ending with white spaces in:
cc/trees/layer_tree_host_impl_unittest.cc:379
R=maruel@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/22837004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@217405 0039d316-1c4b-4281-b951-d872f2087c98
git cl was using system git cl.
- replace git cl status with GIT_CL_STATUS var
- add tested depot_tools to tip of PATH.
R=maruel@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/20889002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@214021 0039d316-1c4b-4281-b951-d872f2087c98
Convert monkey patching to class inheritance for OptionParser usage in:
commit_queue.py, gclient.py and trychange.py.
Monkey patching confuses the hell out of pylint.
R=jochen@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/19552004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@213423 0039d316-1c4b-4281-b951-d872f2087c98
Add an option in DEPS files to clone a project into a temp dir
and then copy into expected final dir. This allows checking out a
git repo into a folder which is non-empty. It is useful for projects
that are embedded in src/ but want to specify the revision of
src/ in the embedded project (such as android private).
BUG=165280
Review URL: https://chromiumcodereview.appspot.com/19359002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@212720 0039d316-1c4b-4281-b951-d872f2087c98
We can use this to evaluate the usefulness of making hooks run in parallel.
Review URL: https://chromiumcodereview.appspot.com/18851005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@211446 0039d316-1c4b-4281-b951-d872f2087c98
Sometimes we wish to pull in a complicated dependency but want to
suppress or replace one or more of the hooks rules. Say for example
we want to use a different way of generating the projects, or
have a different set of landmine expectations.
Here we add a custom_hooks section mirroring custom_deps to allow us to
override sections we have identified in the DEPS file. To do so,
we add an optional name to the elements of the hooks list, and overwrite
those whose name matches.
Conventions between included DEPS and the .gclient as to the meanings of the
name are equivalent to the meaning of the customized deps, and so do not
benefit from further structure or definition.
BUG=None
TEST=local unit test
Review URL: https://chromiumcodereview.appspot.com/17742004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@210573 0039d316-1c4b-4281-b951-d872f2087c98
--no-pager looks ugly in logs and output and makes commands harder to
read. Setting an environment variable is better.
This is a followup to https://chromiumcodereview.appspot.com/14104005/
which added --no-pager.
Review URL: https://chromiumcodereview.appspot.com/18173003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@210503 0039d316-1c4b-4281-b951-d872f2087c98
Instead of cloning straight into place, clones are made to a global cache dir,
and then local (using --shared) clones are made from the cache to the final
resting place. This means the 'final' clones are full repos with no shenanigans,
meaning that branches, commits, etc. all work, which should allow the rest of
the gclient ecosystem to work without change as well.
The primary benefit is, of course, reduced network IO, and a much lower cost for
'clobber' operations (assuming we don't clobber the cache). It also means that
a given bot can have a greater number of checkouts, since the entire git history
will only be stored once per machine, instead of once per checkout.
R=dpranke@chromium.org, szager@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/18328003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@210024 0039d316-1c4b-4281-b951-d872f2087c98
This will be used on the bots when calling apply_patch, which commits to git
repos.
R=dpranke@chromium.org,szager@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/18006002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@209040 0039d316-1c4b-4281-b951-d872f2087c98
Previously if one or both of these values were unset, the tests would simply fail
because `git config` returns 1.
R=dpranke@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/18053009
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@208966 0039d316-1c4b-4281-b951-d872f2087c98
It appears to be parsed and ignored, seems like a bug to me.
BUG=None
TEST=local/manual.
Review URL: https://chromiumcodereview.appspot.com/17265012
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@207404 0039d316-1c4b-4281-b951-d872f2087c98
Greatly improves presubmit performance on Blink where there may be rule violations (my test case went from 50s to 5s).
BUG=236206
TEST=new test in presubmit_unittest.py; manual performance test on Mac and Windows after touching hundreds of files in webkit; testing included add/move/edit/delete on both binary and text files.
Review URL: https://chromiumcodereview.appspot.com/15898005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@205275 0039d316-1c4b-4281-b951-d872f2087c98
When a developer runs download_from_google_storage, and they don't have a .boto
file, the tool automatically runs "gsutil config" to create one for them.
Unfortunately, a side consequence is that if a bot runs the script, and it has
a boto file that 403's, then it would run "gsutil config" which moves the .boto
to .boto.bak, and creates an empty .boto file. This should not be the intended
action.
This CL changes so that "gsutil config" is not called, but instead just fails
with a message telling the dev to run that command.
TBR=maruel@chromium.org
Review URL: https://codereview.chromium.org/16072023
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@203824 0039d316-1c4b-4281-b951-d872f2087c98
If ~/.gitconfig contains:
[diff]
renames = "true"
then git diff --name-status will produce output that confuses the parsing
in svm.GIT.CaptureStatus and git diff will produce output that confuses the
patch command on trybots. Passing --no-renames explicitly counteracts this.
Review URL: https://chromiumcodereview.appspot.com/16057018
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@203778 0039d316-1c4b-4281-b951-d872f2087c98
76 columns is fine for most patches but reverts will eat another
4 spaces in indentation. For the sake of revert clarity this reference
line is reduced to 72 columns.
Review URL: https://chromiumcodereview.appspot.com/16152005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@203530 0039d316-1c4b-4281-b951-d872f2087c98
Adds an extra line to the default text description to help judge whether the
76 character recommended limit is being passed.
Review URL: https://chromiumcodereview.appspot.com/15670012
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@203314 0039d316-1c4b-4281-b951-d872f2087c98
Also adds msg to assertions to print current size of data written to stdout.
Review URL: https://chromiumcodereview.appspot.com/16058007
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@202863 0039d316-1c4b-4281-b951-d872f2087c98
bratell@opera.com
hinoka@chromium.org
maruel@chromium.org
erikchen@google.com
ukai@chromium.org
scottmg@chromium.org
iannucci@chromium.org
stip@chromium.org
szager@chromium.org
borenet@google.com
rmistry@google.com
tyoshino@chromium.org
cyrille@nnamrak.org
ikarienator@chromium.org
dpranke@chromium.org
agable@chromium.org
danakj@chromium.org
ilevy@chromium.org
petermayo@chromium.org
nick@chromium.org
hinoka@google.com
mcgrathr@chromium.org
alancutter@chromium.org
xusydoc@chromium.org