This reverts commit 149a3b7581.
Reason for revert: warnings are issued if no optional parameters
are passed.
Original change's description:
> Warn when non-inclusive parameters are used
>
> R=apolito@google.com
>
> Bug: 1098560
> Change-Id: Iabc2a5d89a6af1418c1228f3c09eacbbd799b5c7
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2355112
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
TBR=apolito@google.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 1098560
Change-Id: Ib0f188de90121c692f5c0d7271938ee312917fd1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2376042
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Use input_api.version and PRESUBMIT_VERSION to ensure newest versions.
Enable presubmit_support to time/ResultDB wrap each individual
lower-level check in PRESUBMIT.py. The approach taken here is to have
the PRESUBMIT_VERSION = 2.0.0 in PRESUBMIT.py
so that presubmit_support is aware of the new version of PRESUBMIT.py,
then check input_api.version in CheckChangeOnCommit/Upload to ensure
that presubmit_support itself is the new version.
Once these changes are landed, the associated changes in PRESUBMIT.py
(crrev.com/c/2350234) can landed as well.
Change-Id: If6d5e318547bc4f128ffe151387c4dc339ef5dc8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2349979
Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Allows presubmit_support to perform Timing and RDB reporting for each of
the lower-level "CheckXYZ" functions, provided that
ENABLE_NEW_PRESUBMIT_CHECKS is called in the PRESUBMIT.py
file that we are running. I am using the variable to avoid the breakage
caused by the similar previous CL (2332321) which was not backwards
compatible.
In addition, asks for "-realm" with ResultDB so it can maintain the
security boundary.
Bug: 1106943
Change-Id: Ia49e8884e2653bee7de7ed7d254452d55dd5c617
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2341828
Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
This reverts commit dba1d65a19.
Reason for revert: "Check* takes exactly 3 arguments (2 given)" presubmit errors
eg: https://ci.chromium.org/p/chromium/builders/try/chromium_presubmit/933062?
https://crbug.com/1113506
Original change's description:
> [Fixed from older branch] Timing/RDB for lower-level checks
>
> presubmit: Report timing to rdb for individual checks
> Rather than using CheckChangeOnCommit() or CheckChangeOnUpload() as the
> sole entry points for PRESUBMIT.py files, this cl enables presubmit_support.py
> to use any CheckXYZ[OnCommit/Upload] function to serve as an entry point.
>
> This way, we can perform timing and RDB reporting for each of the
> presubmit check functions and have check-specific data.
>
> Bug: 1106943
> Change-Id: Ifdabd68c0904a6d70a828f12de157369c6c1571d
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2332321
> Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Dirk Pranke <dpranke@google.com>
TBR=dpranke@google.com,estaab@chromium.org,ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,saagarsanghavi@google.com
Change-Id: Ibd80f70661d2196f903f0c6b79eb74eda6c94d44
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1106943, 1113506
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2340692
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
presubmit: Report timing to rdb for individual checks
Rather than using CheckChangeOnCommit() or CheckChangeOnUpload() as the
sole entry points for PRESUBMIT.py files, this cl enables presubmit_support.py
to use any CheckXYZ[OnCommit/Upload] function to serve as an entry point.
This way, we can perform timing and RDB reporting for each of the
presubmit check functions and have check-specific data.
Bug: 1106943
Change-Id: Ifdabd68c0904a6d70a828f12de157369c6c1571d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2332321
Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Added an rdb wrapper and use it to time CheckChangeOn{Commit, Upload} from every PRESUBMIT.py file, then reports the results to ResultSink.
Bug: http://crbug.com/1107588
Change-Id: I9bc8a18ba615a76f3e0611eae5b2c4adc5924276
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2242097
Commit-Queue: Saagar Sanghavi <saagarsanghavi@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Garrett Beaty <gbeaty@chromium.org>
Some PRESUBMIT.py scripts set DEFAULT_WHITE_LIST and DEFAULT_BLACK_LIST.
This patch adds support to override those defaults during transition
period.
R=apolito@google.com, ehmaldonado@chromium.org
Bug: 1098562
Change-Id: I5dfff253e61485d354a83f3f509d8a6d4d87dec6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2270717
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Slashes and backslashes are interchangeable on Windows in many contexts.
But not all. In particular, if you add a path to a dictionary and then
try looking it up with an equivalent path that has different slashes
then - not surprisingly - the lookup will fail. Therefore it is
important to always use slashes when populating _paths_to_owners. This
is now done, and documented.
Additionally, while fnmatch.fnmatch handles comparing slash separated
paths to backslash separated paths, fnmatch.translate does not. This is
arguably a bug in fnmatch.translate, or at least something that should
be documented. This bug is worked around by having _fnmatch sanitize
file names to always use slashes, and asserting that the patterns do
not contain backslashes.
With these changes this command:
git cl owners --ignore-current --show-all chrome/browser/BUILD.gn
now correctly gives the same results as this command on Windows:
git cl owners --ignore-current --show-all chrome\browser\BUILD.gn
The modified test fails without the other changes, passes with them.
Bug: 1009104
Change-Id: I416c7131281f00e352c1d2b85c30fcc463417fa5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1915410
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
This reverts commit 684096347b.
Reason for revert:
Breaks git-cl split.
https://bugs.chromium.org/p/chromium/issues/detail?id=1054888
Original change's description:
> Improve git cl split
>
> This CL changes the behavior of `git cl split` to split the change
> by the size of the resulting CLs. For now, this is based on the number
> of bytes changed, and not by the number of changed lines. Depending
> on the shape of change, this may still produce more CLs than expected
> (and possibly more than before).
>
> A future change will switch the split to be based on the number
> of affected lines, and also introduce a mode to base the split
> on the number of affected files.
>
> Bug: 998922
> Change-Id: I49f868972a61b89b426ef9e2ceedc733eacb4350
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1778744
> Commit-Queue: Yannic Bonenberger <yannic.bonenberger@gmail.com>
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>
TBR=fdoray@chromium.org,dpranke@chromium.org,yannic.bonenberger@gmail.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com
Bug: 998922
Change-Id: I466e1245d5f1c934443d850c48c42d3e694e71c6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2080205
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Remove input and output stream, that always pointed to sys.stdin
and sys.stdout.
Add tests for DoPostUploadExecuter.
Bug: 1042324
Change-Id: Ib5d092706a242ae97ec877af5dbceef6db7cbb73
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2079411
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
PRESUBMIT.py scripts are executed in presubmit_support.py using exec().
Since PRESUBMIT.py scripts might not be yet compatible with python 3, we
have to execute presubmit_support.py using python 2.
git_cl.py imports presubmit_support.py, and executes presubmit checks using
presubmit_support as a module. This forces git_cl.py to be executed using
python 2 to maintain compatibility for PRESUBMIT.py scripts.
This change allows git_cl.py to be executed using python 3, while
presubmit_support.py is executed using python 2.
Similar changes for post-submit hooks and git-cl try masters will follow.
Bug: 1042324
Change-Id: Ic3bb1c2985459baf6aa04d0cc65017a1c2578153
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2068945
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Also, rename --all-files to --all_files to be consistent with other
options, and remove reviewers, since it's never set in presubmit_support.
Bug: 1042324
Change-Id: I34fa3162c56e39bcdd52ad1e19058357de934f1c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2067351
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
The `author` parameter is never set.
The `local_description` parameter was only set with the
purpose of getting the local description, so separate it into its
own method and call that instead.
Use the absolute root when getting the list of modified files instead.
Bug: 1042324
Change-Id: I8505172edaacee0656dda98ebbd5f8ebfacb3cef
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2038050
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This is a reland of 1c8026d2f1
Original change's description:
> presubmit_support: Add --all-files options.
>
> Will be set when --all is passed to git-cl presubmit to execute
> presubmit checks over all files on a repo.
>
> Bug: 1042324
> Change-Id: I2ebb7c3ec633f9d3a3774314799aa056bc2919c8
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2003071
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Bug: 1042324
Change-Id: I452d98972f9b2c6d0a212029bac5aa6af57dce40
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2038055
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This reverts commit 1c8026d2f1.
Reason for revert:
Description should be set from gerrit before parsing the changes.
Original change's description:
> presubmit_support: Add --all-files options.
>
> Will be set when --all is passed to git-cl presubmit to execute
> presubmit checks over all files on a repo.
>
> Bug: 1042324
> Change-Id: I2ebb7c3ec633f9d3a3774314799aa056bc2919c8
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2003071
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
TBR=ehmaldonado@chromium.org,apolito@google.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com
Change-Id: I8c455a269cdd6eb3dea9379faf351a04644bd23f
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1042324, 1048885
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2038052
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Will be set when --all is passed to git-cl presubmit to execute
presubmit checks over all files on a repo.
Bug: 1042324
Change-Id: I2ebb7c3ec633f9d3a3774314799aa056bc2919c8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2003071
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Needed by Skia's PostUploadHooks, so that presubmit_support
doesn't depend on git-cl classes.
Bug: 1042324
Change-Id: I02475e346437b26fa8d0d2021ea62754077a2b5b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2007262
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Needed by Skia's PostUploadHooks, so that presubmit_support doesn't
depend on git-cl internal classes.
Bug: 1042324
Change-Id: If8f4e6ea035197667cd9705a4274590c4d11511e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2007670
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This CL changes the behavior of `git cl split` to split the change
by the size of the resulting CLs. For now, this is based on the number
of bytes changed, and not by the number of changed lines. Depending
on the shape of change, this may still produce more CLs than expected
(and possibly more than before).
A future change will switch the split to be based on the number
of affected lines, and also introduce a mode to base the split
on the number of affected files.
Bug: 998922
Change-Id: I49f868972a61b89b426ef9e2ceedc733eacb4350
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1778744
Commit-Queue: Yannic Bonenberger <yannic.bonenberger@gmail.com>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Only gerrit_url is needed to initialize gerrit_obj.
This will be needed for git cl to call presubmit support
as a script.
Bug: 1042324
Change-Id: I79d78fb67465f989755bf8c8ceb48d665464fa05
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2005050
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
On Windows, scripts on the current directory take precedence over PATH, so
that when testing depot_tools on Windows, calling `vpython.bat` will
execute the copy of vpython of the depot_tools under test instead of the
one in the bot.
As a workaround, we run the tests from the parent directory instead.
Change-Id: I5187943b6cf7499d50b3f1a2ec906a5e8fd84835
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1910448
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
The problems on crbug.com/1017367 should have been fixed by crrev.com/c/1880013
Bug: 1009814
Change-Id: Ia98304ca983b4e7e293fedb9df1fee589de58ba9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1887714
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Exceptions other than OSError are not surfaced.
This caused errors like this to be printed, but not block presubmit,
allowing bugs to sneak in.
Exception in thread Thread-8:
Traceback (most recent call last):
File "C:\b\s\w\ir\cipd_bin_packages\cpython\bin\Lib\threading.py", line 801, in __bootstrap_inner
self.run()
File "C:\b\s\w\ir\cipd_bin_packages\cpython\bin\Lib\threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "C:\b\s\w\ir\kitchen-checkout\depot_tools\presubmit_support.py", line 199, in _WorkerFn
result = self.CallCommand(test)
File "C:\b\s\w\ir\kitchen-checkout\depot_tools\presubmit_support.py", line 170, in CallCommand
p = subprocess.Popen(cmd, **test.kwargs)
File "C:\b\s\w\ir\kitchen-checkout\depot_tools\subprocess2.py", line 143, in __init__
super(Popen, self).__init__(args, **kwargs)
File "C:\b\s\w\ir\cipd_bin_packages\cpython\bin\Lib\subprocess.py", line 390, in __init__
errread, errwrite)
File "C:\b\s\w\ir\cipd_bin_packages\cpython\bin\Lib\subprocess.py", line 640, in _execute_child
startupinfo)
TypeError: environment can only contain strings
https://logs.chromium.org/logs/infra/buildbucket/cr-buildbucket.appspot.com/8898840708364523888/+/steps/presubmit/0/stdout
Change-Id: I34e65d8c0050eed7ed26fd782e0a5dc8616f30f7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1877051
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
On windows on Python 2, only strings can be environment variables, but
literals are unicode, so we need to convert to strings before setting
env vars.
Change-Id: I40e8e6ae4f5497d56efa568153eb65392dbd85e6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1875108
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Some of the modules imported by presubmit_support are unused, and some of those (cPickle, cStringIO) are not available on Python3, so remove them.
Also expose urllib2 via urllib_request and urllib_error to improve compatibility with Python3.
Bug: 1009814
Change-Id: I9f758df4270cc039df092e814c981a7a87b2c350
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1866170
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Ran:
vi $(git grep --name-only iteritems | grep -v third_party)
vi $(git grep --name-only itervalues | grep -v third_party)
vi $(git grep --name-only 'print ' | grep -v third_party)
and edited the files quickly with adhoc macros. Then ran in recipes/:
./recipes.py test train
There was only a small subset of files that had been updated to use
six.iteritems() and six.itervalues(). Since the dataset size that is
being used in gclient is small (pretty much always below 200 items),
it's better to just switch to .items() right away and take the temporary
performance hit, so that we don't need to come back to rewrite the code.
Recipe-Nontrivial-Roll: build
Bug: 984182
Change-Id: I5faf11486b66b0d73c9098ab0f2ce1b15a45c53e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1854900
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Marc-Antoine Ruel <maruel@chromium.org>
Don't overwrite passed kwargs. Make a copy instead.
Otherwise, subsequent calls with the same kwargs will have
stdin set to subprocess.PIPE.
Bug: 984182
Change-Id: I358ffa1951e8b42486e0ac3a0d3d587a93c6dc4e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1769405
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Add run_on_python2 and run_on_python3 to allow to be tests on
either only python2, only python3 or both.
Bug: 984182
Change-Id: I87d3b67412129cf3a8e627cd6bc97daa81147657
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1758849
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: smut <smut@google.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Adds functionality to presubmit api to return error,
notification and warning messages caught in
presubmit_support.py in the form of a json.
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Bug:971895
Change-Id: I42a3df3994077342216d002381b6135012b4334c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1666250
Commit-Queue: Debrian Figueroa <debrian@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Ran "2to3 -w -n -f except ./".
The scripts still work with Python 2.
There are no intended behaviour changes.
Bug: 942522
Change-Id: Ifa274cb83f74cfa8ce092fffbb88f3ab5309e72c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1607841
Commit-Queue: Raul Tambre <raul@tambre.ee>
Auto-Submit: Raul Tambre <raul@tambre.ee>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
This caused compatibility issues, because the executor script (presubmit_support.py) asks the parser for Python 3-style print functions and inherits this to the presubmit scripts it runs, of which some still use the old style statements.
Bug: 961783
Change-Id: I545d778991a53f484bce6c7e29089e0484dc19ac
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1605917
Auto-Submit: Raul Tambre <raul@tambre.ee>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
Ran "2to3 -w -n -f print ./" and manually added imports.
Ran "^\s*print " and "\s+print " to find batch/shell scripts, comments and the like with embedded code, and updated them manually.
Also manually added imports to files, which used print as a function, but were missing the import.
The scripts still work with Python 2.
There are no intended behaviour changes.
Bug: 942522
Change-Id: Id777e4d4df4adcdfdab1b18bde89f235ef491b9f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1595684
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
Auto-Submit: Raul Tambre <raul@tambre.ee>
Since python2 tempfile does not have TemporaryDirectory, we need to either use temporary_directory in gclient_utils or implement it by ourselves. If we chose latter, we also need to implement rmtree for Windows. Considering that, I suppose using gclient_utils should be much easier.
Bug: 939959
Change-Id: Ife21a2bc297ac61a4c1940b06df937c293961ae7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1524815
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Commit-Queue: Yoshisato Yanagisawa <yyanagisawa@chromium.org>
Auto-Submit: Yoshisato Yanagisawa <yyanagisawa@chromium.org>
The function we use with gclient_utils has moved to gclient_paths.
However, I am not sure gclient_utils is used by others or not.
Additional change:
- sorted testMembersChanged members for ease of understanding what is
different.
Bug: 939959
Change-Id: I8b822e26afc73b39d2a33c5e07779c9c14cbd394
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1521850
Commit-Queue: Yoshisato Yanagisawa <yyanagisawa@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Fumitoshi Ukai <ukai@chromium.org>
The flag value was not being propagated correctly.
Bug: 819774
Change-Id: I42519e1c84704b9a4a613005d3441b7ee12ea427
Reviewed-on: https://chromium-review.googlesource.com/1142533
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Let me make them exported via InputAPI to be used by our repo.
Change-Id: I1a0c7f49ab5a3c00b2617062444d67bbb9600837
Reviewed-on: https://chromium-review.googlesource.com/1117958
Commit-Queue: Yoshisato Yanagisawa <yyanagisawa@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Sometimes 'cwd' has already been specified, for example by GetPythonUnitTests.
Don't overwrite it if it has been.
TBR=agable@chromium.org
Bug: 829084
Change-Id: Ide65c2811bcdd2628d4e886227d634dc66bfd297
Reviewed-on: https://chromium-review.googlesource.com/1038665
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This will allow to ensure that fidl and mojom files have proper
copyright header.
Bug: 831384
Change-Id: I5e41bbf27f2e3f2c6749a52de7463d651d033b81
Reviewed-on: https://chromium-review.googlesource.com/1036606
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Sergey Ulanov <sergeyu@chromium.org>
We recently moved many files in third_party/WebKit to
third_party/blink. DEFAULT_BLACK_LIST should not block third_party/blink
as well as third_party/WebKit.
Bug: 836555
Change-Id: I5b3a3187f976b011c8efc8bf60635d6a5af1b56b
Reviewed-on: https://chromium-review.googlesource.com/1029562
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Kent Tamura <tkent@chromium.org>
Currently all tests in a PRESUBMIT.py file are run in parallel, but not
all tests across PRESUBMIT.py files.
This introduces a flag that will allow presubmit to run all tests across
PRESUBMIT files in parallel.
Bug: 819774
Change-Id: Idd3046cb3c85e9c28932a9789ba7b207a01d9f99
Reviewed-on: https://chromium-review.googlesource.com/994241
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This reverts commit c7d0b34084.
Reason for revert:
See https://logs.chromium.org/v/?s=infra%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8950238735438551408%2F%2B%2Fsteps%2Fpresubmit%2F0%2Fstdout
Two go bootstraps executing concurrently and breaking each other. govet and golint presubmit checks both try to bootstrap go at the same time.
Original change's description:
> presubmit support: Run all tests in parallel.
>
> Currently all tests in a PRESUBMIT.py file are run in parallel, but not
> all tests across PRESUBMIT.py files.
>
> This introduces a pool common to all files so we can run all of them.
>
> Bug: 819774
> Change-Id: Ic129af1bc9e6da568fa9fa71827193c6d8ab9af1
> Reviewed-on: https://chromium-review.googlesource.com/973691
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
TBR=agable@chromium.org,ehmaldonado@chromium.org
Change-Id: Ia5b5ae5af8d6cf9bd72388f58ff0f032a4367e10
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 819774
Reviewed-on: https://chromium-review.googlesource.com/994032
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Since no one can upload or land changes from Rietveld anymore,
PRESUBMIT and its support files no longer need to be able to
support it.
Bug: 770408
Change-Id: I4ca6391291e7b0755c78af453c3d006ad3666a17
Reviewed-on: https://chromium-review.googlesource.com/991053
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
Currently all tests in a PRESUBMIT.py file are run in parallel, but not
all tests across PRESUBMIT.py files.
This introduces a pool common to all files so we can run all of them.
Bug: 819774
Change-Id: Ic129af1bc9e6da568fa9fa71827193c6d8ab9af1
Reviewed-on: https://chromium-review.googlesource.com/973691
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
If someone doesn't want to run a check, and that check doesn't
exist... great! We won't be running that check anyway.
R=iannucci
Bug: 770408, 828154
Change-Id: I74478cac9988e21a1fadfb2c9d23dac1697aaa46
Reviewed-on: https://chromium-review.googlesource.com/991093
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Sean McCullough <seanmccullough@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
This reverts commit f3eed0016e.
Reason for revert: Some PRESUBMIT checks still expect
CheckRietveldTryJobExecution to exist.
Original change's description:
> Remove Rietveld support from presubmit
>
> Since no one can upload or land changes from Rietveld anymore,
> PRESUBMIT and its support files no longer need to be able to
> support it.
>
> R=tandrii@chromium.org
>
> Bug: 770408
> Change-Id: I749092b66fdca16d5cef77e8cefc905aa5375b50
> Reviewed-on: https://chromium-review.googlesource.com/693380
> Commit-Queue: Aaron Gable <agable@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
TBR=agable@chromium.org,tandrii@chromium.org
Change-Id: I72e29bd8a9739406f29190adbeb7eb7718ed21cd
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 770408
Reviewed-on: https://chromium-review.googlesource.com/991012
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Since no one can upload or land changes from Rietveld anymore,
PRESUBMIT and its support files no longer need to be able to
support it.
R=tandrii@chromium.org
Bug: 770408
Change-Id: I749092b66fdca16d5cef77e8cefc905aa5375b50
Reviewed-on: https://chromium-review.googlesource.com/693380
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Currently in LUCI `git cl` is invoked on chromium_presubmit using vpython. This
means that it operates under the depot_tools .vpython environment (which is
blank).
Some features in presubmit_support allow invocation of python subprocesses, and
previously they would use `sys.executable` (especially on windows, but
occasionally on non-windows as well).
In non-vpython environments, this just picks up the system python and whatever
modules happen to be installed there. Some python unittests work around this on
non-windows systems by having a shebang line which explicitly invokes vpython.
This CL changes presubmit_support so that invocations of 'python', or executions
of '.py' scripts will always end up invoking vpython. In the best case, this will
pick up the invoked scripts' vpython environment. In the worst case, this will
invoke the script under an empty vpython environment (aka "stock python").
R=dpranke@chromium.org, jbudorick@chromium.org, tandrii@chromium.org, tikuta@chromium.org
Bug: 821669
Change-Id: I5d2d5dfd0364022d56833c2c8af4983553a29c7a
Reviewed-on: https://chromium-review.googlesource.com/961865
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
This is a reland of 9219d35688
The original was reverted due to a typo (core,quotePath instead
of core.quotePath). This version is fixed.
Original change's description:
> Use core.quotePath=false when git is listing files
>
> This prevents git from putting quotes around some file names
> (those that have astral-plane characters) and not around others.
>
> R=maruel
>
> Bug: 792302
>
> Change-Id: I3b6a6b36c4720116de811b40177b59aa25c263db
> Reviewed-on: https://chromium-review.googlesource.com/815454
> Commit-Queue: Aaron Gable <agable@chromium.org>
> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
Bug: 792302
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Change-Id: I28d2260948aaf63bd865888c2f60e4cdee9aea48
Reviewed-on: https://chromium-review.googlesource.com/822990
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This reverts commit 9219d35688.
Reason for revert: unfortunately this says "core,quotePath" and since it includes recipe changes, we need something that the roller can munch on :(
Original change's description:
> Use core.quotePath=false when git is listing files
>
> This prevents git from putting quotes around some file names
> (those that have astral-plane characters) and not around others.
>
> R=maruel
>
> Bug: 792302
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Change-Id: I3b6a6b36c4720116de811b40177b59aa25c263db
> Reviewed-on: https://chromium-review.googlesource.com/815454
> Commit-Queue: Aaron Gable <agable@chromium.org>
> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
TBR=maruel@chromium.org,agable@chromium.org
Change-Id: I226388f19024403240a1443eb2b878b9293220e1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 792302
Reviewed-on: https://chromium-review.googlesource.com/821671
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
This prevents git from putting quotes around some file names
(those that have astral-plane characters) and not around others.
R=maruel
Bug: 792302
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Change-Id: I3b6a6b36c4720116de811b40177b59aa25c263db
Reviewed-on: https://chromium-review.googlesource.com/815454
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
Sometimes, InputApi users need to create temporary files, write to them and
pass them to another process, like this:
with input_api.tempfile.NamedTemporaryFile() as f:
f.write('foo')
input_api.subprocess.check_output(['/path/to/script',
'--reading-from', f.name])
While this works fine on Unix, on Windows subprocess cannot open and read
the file while we have it open for writing.
To work around this, we now offer a CreateTemporaryFile() that wraps a call
to tempfile.NamedTemporaryFile(delete=False), and we then take care of
removing all files created this way at the end of a presubmit run.
The idea is for users to do something like this:
with input_api.CreateTemporaryFile() as f:
f.write('foo')
f.close()
input_api.subprocess.check_output(['/path/to/script',
'--reading-from', f.name])
with the temporary file being removed automatically in a transparent fashion
later.
Bug: 780629
Change-Id: I0d705a5d52928a43f39a51f94a2c48d277bd5ced
Reviewed-on: https://chromium-review.googlesource.com/758637
Commit-Queue: Raphael Kubo da Costa (rakuco) <raphael.kubo.da.costa@intel.com>
Reviewed-by: Aaron Gable <agable@chromium.org>
To mention that it should not be used for accessing files.
Change-Id: I01ea79134bfc1f394c1791d47eb6fdb798358941
Reviewed-on: https://chromium-review.googlesource.com/750646
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Commit-Queue: agrieve <agrieve@chromium.org>
This reverts commit 873c28d175.
Reason for revert: Broken on Windows (crbug.com/762389)
Original change's description:
> Capture ctrl-c in presubmit multiprocessing pool
>
> Presubmit spins up lots of multiprocessing processes to run
> each individual test. If you cancel your presubmit run with
> <ctrl>+c, that signal gets passed through to each of those,
> which then raises its own KeyboardInterrupt, and prints its
> own stacktrace.
>
> This change has each member of the multiprocessing pool instead
> exit gracefully (albeit with an error code) so that only the
> parent process prints its stacktrace.
>
> R=michaelpg@chromium.org
>
> Bug: 635196
> Change-Id: If9081910a359889a43bc1b72c91a859ebe37a1d6
> Reviewed-on: https://chromium-review.googlesource.com/651764
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Aaron Gable <agable@chromium.org>
TBR=iannucci@chromium.org,agable@chromium.org,michaelpg@chromium.org
Change-Id: Ib8e5b2f59b0060dfbfbeba348e211db292318b3b
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 635196
Reviewed-on: https://chromium-review.googlesource.com/653434
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
Presubmit spins up lots of multiprocessing processes to run
each individual test. If you cancel your presubmit run with
<ctrl>+c, that signal gets passed through to each of those,
which then raises its own KeyboardInterrupt, and prints its
own stacktrace.
This change has each member of the multiprocessing pool instead
exit gracefully (albeit with an error code) so that only the
parent process prints its stacktrace.
R=michaelpg@chromium.org
Bug: 635196
Change-Id: If9081910a359889a43bc1b72c91a859ebe37a1d6
Reviewed-on: https://chromium-review.googlesource.com/651764
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
This gives PRESUBMIT equal support for all the
gerrit-style footers that we're migrating to.
R=iannucci@chromium.org, tandrii@chromium.org
Bug: 710327,710803
Change-Id: I64b8f39ef923d90ebda7dd191b83d1a7cc87c776
Reviewed-on: https://chromium-review.googlesource.com/506551
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
We want PRESUBMIT to be able to equally support
BUG= tags (old style) and Git-Footer: footers
(new style). This change refactors the way that
the presubmit api gives access to those properties
so that it is easier to add support for equivalent
footers.
It also limits the scope of tags/footers that it
exposes, as code search shows no PRESUBMIT files
that take advantage of any of the more esoteric
ones.
Bug: 710327, 710803
Change-Id: I86f1d6cb2e1f0aff9653ef3fb455e0a6f47acf5d
Reviewed-on: https://chromium-review.googlesource.com/506450
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This CL wholly revamps the way presubmit_support adds
CQ_INCLUDE_TRYBOTS lines in commit descriptions. In
particular, when the CL is being uploaded to Gerrit,
it uses our pre-existing support for manipulating git
footers to make the whole process much simpler.
R=tandrii@chromium.org
Bug: 710547
Change-Id: I5858282a44c590f131021fa3820f1cb3f70ef620
Reviewed-on: https://chromium-review.googlesource.com/487831
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This makes the library behave more like what we'd expect,
while still allowing certain function to specify that
returning 204 or 404 is expected/acceptable behavior.
Change-Id: If3ce5598d1603819ee97aaeab0072a9e786ed96d
Reviewed-on: https://chromium-review.googlesource.com/481043
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
Gerrit sometimes returns a full response json object at
the same time as returning a non-200 status code. This
refactor makes it easier for calling code to request
access to that object and handle error cases on its own.
The original version of this commit had a bug where
ReadHttpResponse properly set the default value for
accept_statuses, but all calls which came through
ReadHttpJsonResponse were setting None instead.
Bug: 710028
Change-Id: I8cee435d8acd487fb777b3fd69b5e48e19d2e5a3
Reviewed-on: https://chromium-review.googlesource.com/481060
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
This reverts commit 6d7ab1bfe5.
Reason for revert: Stacktrace:
File "/s/depot_tools/gerrit_util.py", line 816, in GetAccountDetails
return ReadHttpJsonResponse(conn)
File "/s/depot_tools/gerrit_util.py", line 376, in ReadHttpJsonResponse
fh = ReadHttpResponse(conn, accept_statuses)
File "/s/depot_tools/gerrit_util.py", line 365, in ReadHttpResponse
if response.status not in accept_statuses:
TypeError: argument of type 'NoneType' is not iterable
Original change's description:
> Refactor ReadHttpResponse to be error-friendlier
>
> Gerrit sometimes returns a full response json object at
> the same time as returning a non-200 status code. This
> refactor makes it easier for calling code to request
> access to that object and handle error cases on its own.
>
> Bug: 710028
> Change-Id: Id1017d580d2fb843d5ca6287efcfed8775c52cd6
> Reviewed-on: https://chromium-review.googlesource.com/479450
> Commit-Queue: Aaron Gable <agable@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
>
TBR=agable@chromium.org,tandrii@chromium.org,chromium-reviews@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
Change-Id: Ia9d9ce835e207a32e7cc8ee35c0cf40c823c7b78
Reviewed-on: https://chromium-review.googlesource.com/481059
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
Gerrit sometimes returns a full response json object at
the same time as returning a non-200 status code. This
refactor makes it easier for calling code to request
access to that object and handle error cases on its own.
Bug: 710028
Change-Id: Id1017d580d2fb843d5ca6287efcfed8775c52cd6
Reviewed-on: https://chromium-review.googlesource.com/479450
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This allows for having some global comments such as timezones or
long-term unavailability.
The comments go into build/OWNERS.status (that way, they should be
available in all repos that map in build/ for the gn config files).
The local can be overwritten in codereview.settings.
The format is
email: status
Comments (starting with #) are allowed in that file, but they're ignored.
BUG=694222
R=dpranke@chromium.org
Change-Id: I49f58be87497d1ccaaa74f0a2f3d373403be44e7
Reviewed-on: https://chromium-review.googlesource.com/459542
Commit-Queue: Jochen Eisinger <jochen@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Josip Sokcevic
Saagar Sanghavi
Ben Pastene
Aaron Gable
Bruce Dawson
Erik Staab
Edward Lemur
Edward Lesmes
Yannic Bonenberger
Marc-Antoine Ruel
Dan Beam
Debrian Figueroa
Raul Tambre
Yoshisato Yanagisawa
Caleb Rouleau
Sergey Ulanov
Kent Tamura
John Budorick
Robert Iannucci
Daniel Bratell
Mun Yong Jang
Daniel Cheng
Raphael Kubo da Costa
Andrew Grieve
Dan Jacques
Takeshi Yoshino
Jeremy Roman
Kenneth Russell
Jochen Eisinger
Lei Zhang