continuation of: https://codereview.chromium.org/11664024
Moved it from chrome/trunk/src/build to depot_tools/
BUG=153360
TEST=two unittests included in tests/
For end-to-end testing, check out a large directory. Run
find . -name .svn -prune -o -size +1000k -type f -print0 | upload_to_google_storage.py -b chrome-artifacts -0 -
(replacing chrome-artifacts with an upload-able bucket)
to test upload
run "find . -name .svn -prune -o -size +1000k -type f -print0 | xargs -0 rm" to remove the files uploaded. Check that the large binary files have been removed
run "download_from_google_storage.py -r -d -b chrome-artifacts ." to download the files again.
Review URL: https://chromiumcodereview.appspot.com/12042069
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@187951 0039d316-1c4b-4281-b951-d872f2087c98
This will replace the hack in commit-queue/verification/presubmit_shim, and will
be used on the presubmit trybot.
R=maruel@chromium.org
BUG=
Review URL: https://codereview.chromium.org/12481002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@186922 0039d316-1c4b-4281-b951-d872f2087c98
Until an issue is uploaded to Rietveld, we don't know the official
email address to use for an owners check. There are three ways to fix
this: we could attempt to log in to rietveld prior to doing the check
and extract the address to use, or we could use ~/.last_codereview_email_address,
or we can use the email address we can determine from the checkout.
All three options have flaws; the first is particularly awkward since there
doesn't seem to be a good way to fetch the email without posting an issue.
The second is flawed if we use different addresses for different repos,
and the third is flawed if the checkout's email address is different from
the rietveld address, or if it is anonymous.
However, since this is only being used for owners checks (in this case),
anonymous checkouts probably don't matter, and hopefully the cases where
the email addresses differ are rare.
R=maruel@chromium.org
BUG=118388, 150049
Review URL: https://chromiumcodereview.appspot.com/12377023
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@186259 0039d316-1c4b-4281-b951-d872f2087c98
Right now we require approval from someone, and we require an owner
approval, but we don't require an approval from an owner *other than
the patch other*. It's conceivable that we might want this, so
I am making this a configurable argument to the presubmit check.
This will also be needed to ensure that we don't suggest you as an
owner for your own patches, when we actually know who you are.
R=maruel@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/12326151
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@185294 0039d316-1c4b-4281-b951-d872f2087c98
Currently, when we run the OWNERS check, we print the list of directories
that contain the relevant OWNERS files for any modified files in a change
still needing approval.
This has two problems:
1) if we bubble all the way up to the top level OWNERS, we print "" instead of
"src/" or something more useful (bug 157191)
2) for OWNERS files that contain per-file set-noparent entries (like changes to IPC messages), this can be really confusing because an owner of other stuff in the directory might've approved things already.
This change will now print the list of files in the CL that are still unapproved.
This might be a lot more verbose (since you get N lines rather than 1 for N files in a given directory), but hopefully it'll be clearer in the two cases above.
Also, this change takes care of some lingering clean-up in the code to rename some methods to be clearer.
R=maruel@chromium.org
BUG=157191
Review URL: https://chromiumcodereview.appspot.com/12314044
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@184219 0039d316-1c4b-4281-b951-d872f2087c98
The Android WebView code includes several manually-maintained .mk files
that are included by the Android build system. These contain some fairly
long lines as they need to refer to deep pathnames, which means the CQ
cannot be used as it always triggers a presubmit warning.
Allow .mk files to have 200 columns per line instead, which should be
enough for the long paths that are used. Add a test case for this.
BUG=
Review URL: https://chromiumcodereview.appspot.com/12252067
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@182733 0039d316-1c4b-4281-b951-d872f2087c98
previously we would return "*" as one of the suggested owners when
a CL included a file that anyone could approve. If the change had
other owners, the "*" was unnecessary, and if the change only included
wildcard-owned files, "*" isn't very helpful, so I've changed the text slightly.
R=maruel@chromium.org
BUG=169168
Review URL: https://chromiumcodereview.appspot.com/11867016
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@177575 0039d316-1c4b-4281-b951-d872f2087c98
The AffectedSourceFiles query matches against files with paths relative
to repository root. On the other hand _FindAllFiles checks white_lists
and black_lists against files paths relative to PRESUBMIT.py
As a result, when a PRESUBMIT.py is not at RepositoryRoot, RunPylint
can misbehave.
I fix this by modifying the white_lists and black_list regexes to
include the relative directory of PRESUBMIT.py.
Also adding some logging to pylint and simplified misc logic.
Review URL: https://chromiumcodereview.appspot.com/11776016
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@175364 0039d316-1c4b-4281-b951-d872f2087c98
This version handles the case where we need to suggest two reviewers
who have overlapping sets of directories they can review.
R=maruel@chromium.org
BUG=76727
Review URL: https://chromiumcodereview.appspot.com/11639028
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@174216 0039d316-1c4b-4281-b951-d872f2087c98
It turns out that we were weighting all possible owners equally,
and picking the last one out of the list. Given the way we traversed
owners files, and given that we got rid of the "set noparent"s, this
meant that we were always suggesting Ben for just about everything.
This change implements a much smarter algorithm that attempts to balance
number of reviewers and closeness to the files under review. The unit
tests added show specific examples and explanations for why things are
chosen the way they are.
R=maruel@chromium.org
BUG=76727
Review URL: https://chromiumcodereview.appspot.com/11567052
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@173784 0039d316-1c4b-4281-b951-d872f2087c98
If you were creating a new OWNERS file that only had per-file owners
in it (and no catch-all owners for the whole directory), then we
would not look for suggested owners in parent directories, and end up
suggesting nothing. See https://chromiumcodereview.appspot.com/11555036/
for the CL that revealed this.
Also, the unit tests were incorrectly using absolute paths in some cases,
making the code less predictable; I've fixed the unit tests and added
a check for this into owners.py (real changes never used absolute paths,
just paths relative to the checkout root).
R=maruel@chromium.org
Review URL: https://chromiumcodereview.appspot.com/11569018
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@173000 0039d316-1c4b-4281-b951-d872f2087c98
We use the fact the dcommit adds a "Committed:" link to the
description. If we wanted to be more strict, we could make
git cl dcommit post a comment on the CL (which would be nice
since it would notify reviewers that the CL was committed).
Note that this is untested as written, because it doesn't appear
that we have either git-cl integration tests, nor a test for
git cl dcommit adding the word "Committed:" so as written, this
presubmit check is prone to failure. Would like suggestions to
improve.
Also adding DoNotSubmit checks and a couple others to
PanPresubmitChecks.
R=maruel@chromium.org,dpranke@chromium.org
BUG=161702
Review URL: https://chromiumcodereview.appspot.com/11421050
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@171158 0039d316-1c4b-4281-b951-d872f2087c98
This can come up when CQ closed an issue but the user's
local branch is still tied to the issue.
BUG=161702
Review URL: https://chromiumcodereview.appspot.com/11348122
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@171153 0039d316-1c4b-4281-b951-d872f2087c98
settings.GetIsGerrit() would invoke logging.basicConfig() and
preemtps the basicConfig for verbose option in parser.parse_args.
R=maruel@chromium.org,ilevy@chromium.org
BUG=162600
Review URL: https://chromiumcodereview.appspot.com/11418171
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@169843 0039d316-1c4b-4281-b951-d872f2087c98
- Many chrome-internal viewvc links do not support https. UpgradeToHttps
was added 10 months ago and continues to generate broken viewvc links.
Reverts part of crrev.com/9214004
BUG=107838
Review URL: https://chromiumcodereview.appspot.com/11412161
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@169815 0039d316-1c4b-4281-b951-d872f2087c98
Multiple presubmit checks may call the same function multiple times, so it's
worth caching the results to speed up the presubmit check run.
Convert presubmit_support, git-cl and gcl to use it.
R=dpranke@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/11280143
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@169726 0039d316-1c4b-4281-b951-d872f2087c98
Currently, if an unversioned directory is present where we would expect
a versioned repository to be, the following error is printed:
Can't update/checkout %s if an unversioned directory is present.
Delete the directory and try again.
If --reset and --delete_unversioned_trees are used, gclient should delete
the unversioned directory in this case.
This problem can be reproduced using the following recipe:
$ rm -rf src/third_party/webrtc/.svn
$ gclient sync -nRftD
BUG=none
TEST=Verify that above error is fixed. Run all smoke tests.
Review URL: https://chromiumcodereview.appspot.com/11366239
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@167942 0039d316-1c4b-4281-b951-d872f2087c98
Added checks:
- Block upload of a diff from sha not in current branch
(this includes tracking against an unrebased local master)
- Block upload of a diff containing shas from origin/master
(should never happen)
- Use explicit calls to git merge-base instead of calling git diff
with "<branch>..." which implicitly uses merge-base.
BUG=157503
Review URL: https://chromiumcodereview.appspot.com/11262057
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@165887 0039d316-1c4b-4281-b951-d872f2087c98
This allows specificying that a target OS list should override the base
list instead of adding to it. This is useful for iOS builds, where the
desired behavior is to pull only the iOS deps, not the union of the Mac
and iOS deps.
Review URL: https://chromiumcodereview.appspot.com/11363036
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@165671 0039d316-1c4b-4281-b951-d872f2087c98
This will let DEPS files specify a higher local bound for recursion.
Note there is an edge case where a dependency can be initially pulled
in with a lower recursion level and then gets ignored subsequently
when a lower level DEPS overrides it's inclusion level and then
includes the same dependency. I do not solve this case, as
I am intending to use this syntax for top level deps files.
BUG=155780
Review URL: https://chromiumcodereview.appspot.com/11146032
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@162464 0039d316-1c4b-4281-b951-d872f2087c98
- On high core, fast machines, jobs=8 is does not offer good
parallelization. Switch to use number of local cpus.
R=maruel@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/11140019
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@162072 0039d316-1c4b-4281-b951-d872f2087c98
Nobody cares about those .orig files and they get in the way on the try server
for some specific license check.
R=cmp@chromium.org
BUG=152507
Review URL: https://chromiumcodereview.appspot.com/10983092
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@159329 0039d316-1c4b-4281-b951-d872f2087c98
The try slaves run 'git revert' to clean up from the previous run. This only runs 'git reset --hard', which does not remove any untracked files, so if a CL adds a new file it will still be present after 'gclient revert'. Adding git clean -f -d removes untracked files (-f) and directories (-d).
Review URL: https://chromiumcodereview.appspot.com/10986032
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@159020 0039d316-1c4b-4281-b951-d872f2087c98
For an unknown reason, 'svn propset svn:executable * foo.sh' doesn't work on
Windows. It is even more awkward that 'svn propset svn:executable . foo.sh'
works just fine, in particular, subversion replaces the value, as long as it's
not an empty string, back to '*'.
R=petermayo@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/10967071
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@158281 0039d316-1c4b-4281-b951-d872f2087c98
Add new test cases to verify it properly handles "-1 +1, 2" and "-1 +1" for
short files.
R=petermayo@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/10894036
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@153906 0039d316-1c4b-4281-b951-d872f2087c98
I recently needed to run presubmit tests in depot_tools, and I was helpfully told:
$ gcl presubmit mychange
... eliding stuff ...
Install google_appengine sdk in /scratch/scratch/chrome/depot_tools_git/testing_support or higher up
This was a lie. If you put google_appengine there, the same presubmit
tools that told you to install it there will fail you for numerous
style violations contained within appengine itself. This change makes
us ignore those by blacklisting google_appengine inside of
testing_support.
Review URL: https://chromiumcodereview.appspot.com/10829293
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@151210 0039d316-1c4b-4281-b951-d872f2087c98
Many, many times this has burned me badly; if you have a dirty index,
git try will just upload your commits, and not the index.
R=nsylvain@chromium.org
BUG=None
Review URL: https://chromiumcodereview.appspot.com/10836180
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@151202 0039d316-1c4b-4281-b951-d872f2087c98
Using urllib for SSL connections when behind a proxy is known to be
broken.
Upstream has fixed this in urllib2 in Python 2.6.3 and newer so
replace uses of urllib for SSL connections with urllib2 methods.
R=maruel@chromium.org
BUG=134165
TEST=gclient sync behind corporate proxy. Submitting this CL with git_cl.
Review URL: https://chromiumcodereview.appspot.com/10825107
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@149742 0039d316-1c4b-4281-b951-d872f2087c98
In some enviroments (eg, OSX zh_CN), use 'svn --version' and splitting string
may get error version string. Use 'svn --version --quiet' to do stable svn
version check, which directly returns version string such as '1.7.0'.
BUG=
TEST=
Review URL: https://chromiumcodereview.appspot.com/10805037
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@147670 0039d316-1c4b-4281-b951-d872f2087c98
git cl dcommit statistics were not using --find-copies-harder, scaring the user
off.
R=cmp@chromium.org
BUG=
TEST=
Review URL: https://chromiumcodereview.appspot.com/10795003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@147343 0039d316-1c4b-4281-b951-d872f2087c98
Otherwise, the files are opened in 'whatever happens to be the current encoding'
which is highly system-dependent. Even if some files are not encoded with utf-8,
the status quo is even worse. So it's worth trying out.
TBR=cmp@chromium.org
BUG=
TEST=
Review URL: https://chromiumcodereview.appspot.com/10697036
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@145306 0039d316-1c4b-4281-b951-d872f2087c98
First, the environment variable for the child process was created but not
specified to subprocess.call().
Second, third_party/logilab/__init__.py tried to initialize itself with
pkg_resources.
TBR=chrisha@chromium.org
BUG=
TEST=
Review URL: https://chromiumcodereview.appspot.com/10582031
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@143111 0039d316-1c4b-4281-b951-d872f2087c98
This CL adds pylint (and its dependencies) to third_party. It plumbs them into presubmit_canned_checks, and exposes a command-line tool to access pylint.
BUG=
TEST=
Review URL: https://chromiumcodereview.appspot.com/10447014
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@143016 0039d316-1c4b-4281-b951-d872f2087c98
Enable copy detection for git-cl upload. This makes
it possible to copy a directory containing many files,
add+commit the new path, and then upload a patch to
Rietveld that shows the files were copied.
In my tests, -C -C was needed to pick up a basic
file copy. -C was not enough. I'm not sure why
exactly. The output generated in the diff looks
like:
sh$ git diff -C -C HEAD~1..HEAD
diff --git a/slave/Makefile b/slave2/Makefile
similarity index 100%
copy from slave/Makefile
copy to slave2/Makefile
...
(This change requires r141676 /
https://chromiumcodereview.appspot.com/10543116/ which
updates upload.py to use --find-copies-harder.)
R=maruel@chromium.org
Review URL: https://chromiumcodereview.appspot.com/10545107
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@141689 0039d316-1c4b-4281-b951-d872f2087c98
The effect of this is to preserve the behavior of `git status`,
i.e., only changes in the top-level repository will be reported.
Updated test expectations; I have no idea what I'm doing.
Review URL: https://chromiumcodereview.appspot.com/10454088
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@140396 0039d316-1c4b-4281-b951-d872f2087c98
Enable copy detection for git-cl upload. This makes
it possible to copy a directory containing many files,
add+commit the new path, and then upload a patch to
Rietveld that shows the files were copied.
In my tests, -C -C was needed to pick up a basic
file copy. -C was not enough. I'm not sure why
exactly. The output generated in the diff looks
like:
sh$ git diff -C -C HEAD~1..HEAD
diff --git a/slave/Makefile b/slave2/Makefile
similarity index 100%
copy from slave/Makefile
copy to slave2/Makefile
...
Review URL: https://chromiumcodereview.appspot.com/10412027
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@140314 0039d316-1c4b-4281-b951-d872f2087c98
So that a command like
"git config rietveld.viewvc-url http://src.chromium.org/viewvc/chrome?view=rev&revision="
works fine.
Fix the description automatically generated for the hooks to be using the same
code than for the one presented to the user.
R=cmp@chromium.org
BUG=
TEST=Tested manually on cygwin, win32, linux
Review URL: https://chromiumcodereview.appspot.com/10447021
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@138874 0039d316-1c4b-4281-b951-d872f2087c98
BUG=None
TEST=Change files that need OWNERS review. Upload the patch. Check that the warning suggests a minimum set of reviewers.
Review URL: https://chromiumcodereview.appspot.com/10222020
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@135619 0039d316-1c4b-4281-b951-d872f2087c98
Add corresponding support to scm.SVN.Revert() to clobber the svn:ignore'd files.
This makes the commit queue remove all the unversioned files, which could have
improved its stability. It failed in practice to improve it but it's still a
good thing to do overall.
R=petermayo@chromium.org
BUG=
TEST=
Review URL: http://codereview.chromium.org/10355014
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@135333 0039d316-1c4b-4281-b951-d872f2087c98
Rietveld self-corrupts its status all the time, there is not point in bailing
just because of that. So just assume it's an 'M' and it'll work fine. In any
case, the diff is properly parsed to detect what was the real operation.
Add unit test that correctly process a deleted file with status:null.
TBR=nsylvain@chromium.org
BUG=
TEST=A corrupted patchset can still be committed with the Commit Queue.
Review URL: http://codereview.chromium.org/10272024
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@134733 0039d316-1c4b-4281-b951-d872f2087c98
This ability is useful for versions of Chromium (i.e. Android) which want to
cross-compile to another platform that has a fixed set of custom dependencies.
Review URL: http://codereview.chromium.org/10127004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@134280 0039d316-1c4b-4281-b951-d872f2087c98
This would spurious try job failures because the gclient revert step would throw
an exception.
R=cmp@chromium.org
BUG=
TEST=
Review URL: http://codereview.chromium.org/10169034
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@133799 0039d316-1c4b-4281-b951-d872f2087c98
Update pylintrc to be near the pylint 0.25.1 default file.
R=cmp@chromium.org
BUG=
TEST=
Review URL: http://codereview.chromium.org/10199016
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@133707 0039d316-1c4b-4281-b951-d872f2087c98
The first approach crrev.com/132446 was reverted due to
Windows build breaks. This approach is more robust because
it actually tests the binary by running the command actually.
BUG=114483
TEST=succeeded on Linux
Review URL: http://codereview.chromium.org/10103024
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@132783 0039d316-1c4b-4281-b951-d872f2087c98
This is useful if git-cl has no way to find out the original svn repo's url
and the user does not wish to configure git-svn and fetch the repository.
Review URL: http://codereview.chromium.org/9969099
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@130372 0039d316-1c4b-4281-b951-d872f2087c98
Enforces that each builder and builder:test association must be in its own list item in PRESUBMIT.py/GetPreferredTrySlaves().
If a user run git try -t foo, trychange.py will now take the list of all the
slaves, will skip any :compile builder, and apply the test filter to them.
Currently, git try -t foo on a chromium checkout throws an exception because of
PRESUBMIT.py files containing test specification.
R=petermayo@chromium.org
BUG=
TEST=
Review URL: http://codereview.chromium.org/9664015
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@125975 0039d316-1c4b-4281-b951-d872f2087c98
Previously, it would apply the test filter only on the last bot.
TBR=dpranke@chromium.org
BUG=
TEST=
Review URL: http://codereview.chromium.org/9634011
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@125622 0039d316-1c4b-4281-b951-d872f2087c98
The previous behavior would cause inproper parsing;
- the bot=bot1 flag would add 'bot1': ['defaulttests'] to the dictionary.
- the testfilter=test1 flag would append 'test1' to the list, causing the value
to look like 'bot1': ['defaulttests', 'test1'].
I think on the long run the best is to keep --testfilter as a user interface but
not on the try server. This effectively stops sending testfilter=foo to the
server.
Add unit test to verify it's working properly. Refactor a bit to make unit
testing simpler.
R=petermayo@chromium.org
BUG=
TEST=
Review URL: http://codereview.chromium.org/9599012
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@125405 0039d316-1c4b-4281-b951-d872f2087c98
rogerta@chromium.org
hinoka@google.com
iannucci@chromium.org
dpranke@chromium.org
sbc@chromium.org
torne@chromium.org
ilevy@chromium.org
jam@chromium.org
ukai@chromium.org
maruel@chromium.org
davidjames@chromium.org
sivachandra@chromium.org
stuartmorgan@chromium.org
srdjan@google.com
szager@chromium.org
lliabraa@chromium.org
gavinp@chromium.org
wittman@chromium.org
joshua.lock@intel.com
jamesr@chromium.org
shouqun.liu@intel.com
dbeam@chromium.org
chrisha@chromium.org
chrisha@google.com
cmp@chromium.org
zork@chromium.org
peter@chromium.org
mukai@chromium.org
kalmard@homejinni.com
bauerb@chromium.org
pam@chromium.org
szager@google.com