Some presubmits can be take orders of magnitude longer than others, but
finding these presubmits is difficult if the time is printed at the end
of the output lines. Moving the time to the start, with enough alignment
for all plausible times, makes scanning the output easy.
Bug: 1309977
Change-Id: I9ba1a599e9825801e1128844a39c7ccafe046f91
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3744130
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Headers currently have the format: `** Presubmit ${name} **`. Make these
more helpful by including the number of total errors there are.
Bug: 1341987
Change-Id: Ib41a133c31568d4264d73c14350f0bbd9590356a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3756169
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
PRESUBMIT.py files should not have CRLF line endings, but if a git repo
is misconfigured this may happen and it is important for the presubmits
to run properly so that they can report on this issue.
In particular, the regex which detects USE_PYTHON3 = True did not work
with CRLF line endings.
It was supposed to handle these, and there is even a comment saying:
# Accept CRLF presubmit script.
However the line after that uses 'rU' as the file open mode in order to
try to get universal newlines, but FileRead ignores the file mode. So,
the replace method is now used instead.
Bug: angleproject:7475, angleproject:4905
Change-Id: I38321176feca089a5796b24756b371709d852243
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3739269
Reviewed-by: Eddie Hatfield <eddiehatfield@google.com>
Reviewed-by: Cody Northrop <cnorthrop@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
The multiprocessing module on Windows has a bug where if you ask for
more than 60 child processes then it will hang. This is related to the
MAXIMUM_WAIT_OBJECTS (64) limit of WaitForMultipleObjects. Other sources
have listed the multiprocessing limit as being 61, or have said that the
the maximum number of objects that can be waited on is actually 63, but
those details don't really matter.
The original fix for this class of issues was crrev.com/c/2785844. This
change extends those fixes to depot_tools, which was missed last year.
This change also updates how PyLint is called by further limiting the
number of jobs to the number of files being processed divided by 10.
This is because there is a significant cost to creating PyLint
subprocesses - each takes about 0.5 s on my test machine. So there needs
to be enough parallelism to justify this.
Patches for PyLint and a bug for cpython are planned.
This will stop PyLint from hanging during presubmits on many-core
machines. The command used to reproduce the hangs and validate the fix
was:
git cl presubmit -v --force --files "chrome/test/mini_installer/*.py"
Prior to this change this command would use (on my many-core test
machine) 96 processes and would hang. How it uses just two processes
because there are only 16 files to analyze.
Output before:
Pylint (16 files using ['--disable=cyclic-import'] on 96 cores)
Output after:
Pylint (16 files using ['--disable=cyclic-import'] on 2 processes)
This is actually not quite true because the hang would prevent the
old message from being displayed.
Bug: 1190269, 1336854
Change-Id: Ie82baf91df4364a92eb664a00cf9daf167e0a548
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3711282
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
While all Chromium PRESUBMIT.py scripts have been running on Python 2
for a long time they continue to invoke child scripts under Python 2.
Part of the reason for slow progress on this transition is that it is
not easy to tell that this is happening, and most developers probably
assume that Python 3 presubmits implies a lack of Python 2.
This change adds a warning when it detects Python 2 scripts being run.
Typical output (edited for clarity) looks like this:
git cl presubmit --files "chrome/updater/tools/*;ppapi/generators/*"
...
Python 2 scripts were run during Python 3 presubmits. Please ask ??? if help is needed in preventing this.
"depot_tools\pylint-1.5" --args-on-stdin from chrome\updater\tools \
"depot_tools\pylint-1.5" --args-on-stdin from chrome\updater\tools \
idl_tests.py from ppapi\generators
If Python 2 scripts launch child scripts, especially if they use
sys.executable, then they will not be reported. However this is a good
thing because it means that the report focuses on the top-level scripts
that drive Python 2 usage.
This change works by modifying vpython.bat to write invocation
information to a text file. The data in this text file is picked up by
presubmit_support.py when it finishes running a set of presubmits.
Bug: 1313804
Change-Id: Ic632b38eae07eca2e02e94358305cc9c998818e9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3699002
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Some presubmit failures result in uncaught exceptions which historically
have halted presubmit runs. This behavior makes it more difficult to
find all failures, measure presubmit times, or detect other behavior
over time. This is particularly problematic when running "git cl
presubmit --all".
As an example, presubmit --all was broken and would not complete from
when crrev.com/3642633 landed until crrev.com/c/3657600. The minimal
repro was this presubmit command:
git cl presubmit --files ui\accessibility\ax_mode.h
This command, or presubmit --all, triggered this error, which halts the
presubmits:
Evaluation of CheckChangeOnCommit failed: [Errno 2] No such file or directory: 'chrome/browser/resources/accessibility/accessibility.js'
Traceback (most recent call last):
File "presubmit_support.py", line 2038, in <module>
sys.exit(main())
File "presubmit_support.py", line 2015, in main
return DoPresubmitChecks(
File "presubmit_support.py", line 1738, in DoPresubmitChecks
results += executer.ExecPresubmitScript(presubmit_script, filename)
File "presubmit_support.py", line 1602, in ExecPresubmitScript
self._run_check_function(function_name, context, sink,
File "presubmit_support.py", line 1640, in _run_check_function
six.reraise(e_type, e_value, e_tb)
File "C:\Users\brucedawson\.vpython-root\e726d2\lib\site-packages\six.py", line 686, in reraise
raise value
File "presubmit_support.py", line 1630, in _run_check_function
result = eval(function_name + '(*__args)', context)
File "<string>", line 1, in <module>
File "PRESUBMIT.py", line 314, in CheckChangeOnCommit
errs.extend(CheckModesMatch(input_api, output_api))
File "PRESUBMIT.py", line 283, in CheckModesMatch
ax_modes_in_js = GetAccessibilityModesFromFile(
File "PRESUBMIT.py", line 245, in GetAccessibilityModesFromFile
for line in open(fullpath).readlines():
FileNotFoundError: [Errno 2] No such file or directory: 'chrome/browser/resources/accessibility/accessibility.js'
With this change the exception is handled and presubmits continue after
printing this message:
Evaluation of CheckChangeOnCommit failed: [Errno 2] No such file or directory: 'chrome/browser/resources/accessibility/accessibility.js', Traceback (most recent call last):
File "presubmit_support.py", line 1630, in _run_check_function
result = eval(function_name + '(*__args)', context)
File "<string>", line 1, in <module>
File "PRESUBMIT.py", line 314, in CheckChangeOnCommit
errs.extend(CheckModesMatch(input_api, output_api))
File "PRESUBMIT.py", line 283, in CheckModesMatch
ax_modes_in_js = GetAccessibilityModesFromFile(
File "PRESUBMIT.py", line 245, in GetAccessibilityModesFromFile
for line in open(fullpath).readlines():
FileNotFoundError: [Errno 2] No such file or directory: 'chrome/browser/resources/accessibility/accessibility.js'
That is, the crucial information regarding the failure is printed, but
subsequent presubmits are not skipped.
Bug: 1309977
Change-Id: I7cfeda85c830e6c8e567c0df3c50f27af1dbe835
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3653978
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Some of the expensive checks when running presubmit --all, such as
CheckStableMojomChanges (~300 s) and CheckAddedDepsHaveTargetApprovals
(~200 s) only look at diffs and are therefore guaranteed to be NOPs when
running presubmit --all or --files=. Passing along the no_diffs state
lets these expensive checks be skipped, thus allowing for faster
iteration times.
Initial testing suggests that (with some supporting changes in the
Chromium repo) this reduces "presubmit --all" times by about ten
minutes, or a bit more than 10%, and additional improvements may be
possible.
Special handling for the no-diffs case also offers a simple way to avoid
presubmit failures that happen whenever all files are flagged as being
changed.
Finally, and perhaps most importantly for having a presubmit --all bot,
when --no_diffs is passed we can treat errors like "Issue wasn't
uploaded" and "Add a description to the CL" as messages, thus making it
possible to have zero presubmit errors when run on origin/main.
Bug: 1320937, 1322936
Change-Id: I0d09dd4aae8fdaa48c8b2f89337441cf96dcff72
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3628368
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Presubmit message types were printed in an inconsistent order, generally
determined by which type was encountered first. This unpredictability
can be confusing when comparing runs. This change enforces a consistent
order, in increasing order of severity. ERRORS always go last so that
they will be the most visible when running presubmits locally.
This also adds a "There were Python %d presubmit errors." message to
the end to make it easier to tell when there were presubmit errors.
This is particularly useful when manually running presubmits.
Bug: 1309977
Change-Id: Ib2b73c7625789bad5b21ae12abf238b746cd11e1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3668724
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
All of the PRESUBMIT.py files in the Chromium repo are running under
Python 3. However "git cl presubmit" also works with other repos where
some PRESUBMIT.py scripts still run under Python 2. This means that
the Python 2 presubmit commit checks step cannot simply be disabled.
That meant that Chromium was paying up to a one-minute cost just to
setup for and look for Python 2 scripts that it doesn't run.
This change runs the Python 3 PRESUBMIT.py scripts first, and keeps
track of whether any were skipped. If none were skipped then the
Python 2 PRESUBMIT.py stage can be skipped.
Note that the child scripts of PRESUBMIT.py scripts may still be run
under Python 2, but that is orthogonal to this change.
Bug: 1313804
Change-Id: Ib65838223f232f1e78058d6a08ea15a89f442310
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3614453
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
AffectedFiles carefully added a slash to dir_with_slash, but then called
normpath - which trims trailing path separators. This meant that
ui/views/PRESUBMIT.py would analyze files in ui\views_content_client,
leading to spooky action at a distance. This was noticed when this
command triggered a presubmit error:
git cl presubmit "--files=ui/views_content_client/*.cc;ui/views/readme.md"
when running presubmit on either file individually found nothing.
This change was tested by getting CheckChangeLintsClean to print the
files that it was asked to analyze, making the behavior quite obvious.
This bug appears to have existed for about thirteen years, but only
triggers in rare cases, and even then the incorrect behavior was almost
impossible to notice.
One of the tests was inadvertently testing the broken AffectedFiles
behavior and another seemed to be requiring it to handle '.'
correctly which I'm not sure we want to support.
Bug: 1309977
Change-Id: Ibdc39981d69664b03448acb228d4ab05b49436f6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3632034
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
The files_to_skip lists in chrome/android/java/src/PRESUBMIT.py were
not working on Windows because they didn't use the verbose [\\\/]
construct for path separators. Fixing this in FilterSourceFile means
that multiple failures may be fixed simultaneously. The only failures
that this is known to fix are a set of AlertDialog.Builder errors that
were showing up only on Windows.
This addresses the issues that were going to be incorrectly fixed in
crrev.com/c/3606128 (now abandoned).
Bug: 1309977
Change-Id: I7a10483448e00df55e2cbf8bff8bad7a5d8db124
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3609117
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
presubmit --all tells the presubmit system that all files are 'modified'
but ChangedContents still goes off to see what changes are present. If
most files were unchanged this was all handled perfectly, but if _all_
files were unchanged then _GitDiffCache would interpret the empty
dictionary of changes as a reason to ask git for diffs, millions of
times. This made some checks take more than 100x as long. The overall
effect on presubmit --all time is not known because I was never willing
to wait the multiple days for them to terminate.
That is, this would take many days to run:
git checkout -b empty -t origin/main
git cl presubmit --all
whereas a single-character change to any file would let this run in
about two hours.
After three weeks of working on presubmits I only hit this twice which
is why it took me so long to realize what the problem was.
Bug: 1309977
Change-Id: Ib280ea386107843b9174d835b0895316a5ed240c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3589900
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
It is easy to get type confusion and end up passing a list as the
message parameter to _PresubmitResult. This error will not be detected
until the end of the run - perhaps hours later - when all evidence of
where the list came from is lost.
This change ensures that the message parameter is a string. If it is not
then the exception that is thrown should allow quick identification of
the problematic code.
This also fixes a presubmit unit test that passed None as the message.
We could support that but I don't think that we should.
Bug: 1309977
Change-Id: Ifb1d5100d47922b0ebd8bb834caa6fbba690b43c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3566436
Reviewed-by: Jochen Eisinger <jochen@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Chromium's presubmits contain a lot of latent errors - presubmit errors
that will trigger the next time a file is modified, but have been
waiting for years. Flushing those out with "git cl presubmit --all"
works poorly because it exercises all presubmits simultaneously, which
is too slow and triggers too many failures. Adding a --files option lets
small areas of the tree or specific file types to be exercised in a
controlled manner.
Bug: 1311697
Change-Id: I36ec6a759a80000d6ed4a8cc218ece327d45f8d6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3559174
Auto-Submit: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
When running git cl presubmit --all which is handy for finding
latent presubmit bugs, the 'No diff found for %s' gets printed many
thousands of times, making actual issues difficult to find. This change
suppresses that message, to make actual issues easier to find.
The message for slow presubmits prints the name of the slow presubmit,
but there are 243 different CheckChangeOnCommit functions, so the name
is actually not sufficient. Therefore this change plumbs through the
path to the script containing the presubmit and prints it.
Similarly, the cpplint message "Done processing %s" doesn't add enough
value.
These changes will make it easier to find the signal in the presubmit
noise.
Bug: 1309977
Change-Id: Iba40b5748266e3296eeb530bb00182db4814aa5d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3556594
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
depot_tools has no versioning. It's hard to know if reported issues were
caused due to outdated depot_tools or actual unresolved bug.
This CL adds basic information about depot_tools version and it's
included in presubmit failure.
R=aravindvasudev@google.com, gavinmak@google.com
Change-Id: If8577c0826063693a7278a57a0cce629d4b1325f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3541061
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
git cl presubmit has --all option which marks all files as modified.
However, that doesn't work well with SCM diff. That is, git diff may not
contain affected file. Instead of throwing an exception in such case,
just return no diff.
R=gavinmak@google.com
Fixed: 808346
Change-Id: Ie1d534b8eebc84bc5206eba7db8057829390fbec
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3485501
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Currently, post upload presubmit hooks are exlusively executed with py2,
regardless of USE_PYTHON3 magic variable. This change adds py3 support
in the same fasion as regular presubmit hooks.
R=aravindvasudev@google.com
Fixed: 1297712
Change-Id: Ib464f8563e4135a63fc48692d27c8692fe1f630b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3469285
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
This includes a few fixes for specific errors, and disables several new
warnings introduced in this version, in order to allow for an incremental migration.
Bug:1262286
Change-Id: I4b8f8fc521386419a3121bbb07edc8ac83170a94
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3413679
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
This reverts commit 22bf605bb6.
Reason for revert: breaks gclient sync
Original change's description:
> Use pylint 2.7 for depot_tools
>
> This includes a few fixes for specific errors, and disables several new
> warnings introduced in this version, in order to allow for an incremental migration.
>
> Bug:1262286
> Change-Id: Ie97d686748c9c952e87718a65f401c5f6f80a5c9
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3400616
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Bug: 1262286
Change-Id: Ieb946073c7886c7bf056ce843a5a48e82becf7a5
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3413672
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
This includes a few fixes for specific errors, and disables several new
warnings introduced in this version, in order to allow for an incremental migration.
Bug:1262286
Change-Id: Ie97d686748c9c952e87718a65f401c5f6f80a5c9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3400616
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Gitwatch supports closing bugs with a FIX(ES|ED|ING)? tag in the
commit message, but we only check BUG to ensure a bug is
specified, add the fix fields to BugsFromDescription so that presubmits
will still pass if only eg: FIXES=b:1234 is given.
Gitwatch reference:
http://shortn/_LGPgBjgi3A
FIXED=b:203812728
TEST=presubmit_unittest.py doesn't fail more than it already does
Change-Id: I6afc38c786e281dcefb4d359bb9212ebe5980ed6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3237598
Auto-Submit: Sean McAllister <smcallis@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
When presubmit checks fail, we want to print the error message from the
underlying check/tool. We use _PresubmitResult objects to collect these
diagnostics and output them to the console and json files.
This change ensures that the string values passed to the
_PresubmitResult constructor will be converted, as needed, to 'str'
instances. This way, we will avoid encoding errors and make sure to
print the actual messages instead of repr(message).
SHERIFFS: this patch breaks backwards compatibility of presubmit error
handling but we don't expect issues in downstream projects. Please
revert if downstream presubmit checks start failing spuriously.
Bug: 1225052
Change-Id: If7f5c99cb5d730c1bfbd6d108b912f77b5f2455c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3037262
Commit-Queue: Tom McKee <tommckee@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
This CL adds a new field to the codereview.settings file used by
git_cl for project-wide defaults. If `USE_PYTHON3` is set to True,
then we will run the PRESUBMIT checks under Python3 by default
instead of Python2, unless the PRESUBMIT.py file contains
`USE_PYTHON3 = False` on a line by itself
(as opposed to now, where we'll use Python2 by default unless the
file contains `USE_PYTHON3 = True`).
This will allow us have Python3 be the default for new files
and to eventually eliminate any uses of `USE_PYTHON3` from the
individual presubmit files. Of course, you will have to go in and
explicitly add `USE_PYTHON3 = False` to any Py2-only files prior
to enabling this.
Bug: 1207012
Change-Id: Id8ec45b43940e5bcffeee196398c711c541e733e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2917747
Reviewed-by: Nico Weber <thakis@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@google.com>
Currently, GitChange.AllFiles returns a list of byte[]. This is
inconsistent with py2 and may cause issues when byte[] is used when
string is expected.
Change-Id: I3e73bc5f71aa787f55357b8139cdeeabee056047
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2946934
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
When presubmit checks create warnings that should raise a prompt
whether or not to continue, the output wasn't being flushed
properly under python3.
Bug: 1213316
Change-Id: Ibb4b4d965d49f0cd65ccc0737282e5f5b5f58004
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2921323
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Dirk Pranke <dpranke@google.com>
Commit 6fc394f added some diagnostics, but it didn't quite work out the
way it was planned.
Bug: 1210746
Change-Id: Ia4f30606e87d3f8b1362441bd74bb7ed0f5ca886
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2928771
Auto-Submit: Greg Thompson <grt@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
If a presubmit happened to generate a crash, the logic we have
for reporting the presubmit check times would call `six.reraise()`
incorrectly, leading to a second crash.
This CL fixes the invocation of the second crash, and also adds
an error message to help users tripping over one possible source
of the first crash (reading a binary file as text).
Bug: 1210746
Change-Id: Ic46f38901b6acf2055b3feb7272dc751dc69037c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2921322
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@google.com>
Presubmits when uploading occasionally take a very long time - over a
minute - and after the fact there is no easy way to know why. This makes
fixes to slow presubmits take longer.
This change builds on crrev.com/c/2532895 to print a message when any
individual presubmit function takes longer than ten seconds. During
normal usage this is a NOP but it will presumably find the long poles
when presubmits are running particularly slowly.
This is similar to gclient runhooks where any hooks that take longer
than ten seconds have their run time printed.
Change-Id: If57ed35d7a7d221f6380e9b97cf72af56f75e441
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2911594
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
CheckForWindowsLineEndings is a Chromium presubmit to make sure that
\r\n line endings (Windows line endings) don't get committed. Among
other things these line endings mess up the license checks, leading to
cryptic errors.
The problem is that ChangedContents() was stripping line endings, making
any checking for them futile. This change adds an option to preserve
line endings, to be used by Chromium's CheckForWindowsLineEndings.
It's not clear how long this has been broken.
See also crrev.com/c/2911914 which must land after this.
Bug: 801033
Change-Id: I7cdb9b3581788624f9eca081da43bb070ee412a1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2910488
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
This fixes CL:2880843 to use a module already present in vpython.
Bug: 1206782
Change-Id: If4158a62011e043e89f40f368c9c644434df6614
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2901758
Commit-Queue: Allen Webb <allenwebb@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
This is a followup to CL:2797935 to make it easier to debug exceptions.
Bug: 1206782
Change-Id: Ibb649e94fd562919737de152e3203ca6b02b38ad
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2880843
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Allen Webb <allenwebb@google.com>
This CL adds support for running PRESUBMIT.py under either Python2
or Python3 as specified in each PRESUBMIT.py file.
To run the checks under Python3, the PRESUBMIT.py file must contain
a line exactly matching "^USE_PYTHON3 = True$". If the file
does not contain this string, the checks will run under Python2.
Different PRESUBMIT.py files in a single CL may thus contain
a mix of 2- and 3-compatible checks, but each individual file will
only be run in one or the other (it doesn't likely make sense to run
them in both by default).
Bug: 1157663
Change-Id: Ic74977941a6519388089328b6e1dfba2e885924b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2832654
Commit-Queue: Dirk Pranke <dpranke@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
This is a reland of 2cf835a9ba
URL-encoded repo path, so chromium/src becomes chromium%2Fsrc
as Gerrit expects
Original change's description:
> presubmit: Skip owners checks if code-owners plugin is enabled.
>
> If code-owners plugin is enable for the repo, skip owners check on
> commit, and skip checking owners format, as that will be done by
> the plugin.
>
> Change-Id: I1663baef4f0f27b00423071343fe740f6da50ce7
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2727131
> Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Change-Id: I3038590f3a92cbf7b6dc0ba6eb47f72593a2ccf7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2775840
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
This is a reland of 36de4be91e
Should be fixed after cl/363445481
Original change's description:
> presubmit: Use code-owners plugin if available.
>
> Change-Id: I56734de985731d007360a4a4e65f2b16de28b69a
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2753894
> Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Change-Id: Id7d5c6c3bd0a246e3400ca99584bb41a92853d16
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2774218
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Bruce Dawson
Gavin Mak
Josip Sokcevic
dpapad
Louis Romero
Andrew Grieve
Aravind Vasudevan
Sean McAllister
Tom McKee
kyle Ju
Dirk Pranke
Greg Thompson
Allen Webb
Erik Staab
Edward Lesmes
Edward Lesmes