Python (prior to 3.8) treats meaningless string escape sequences as if
they were a slash followed by the character. That is, '\w' == '\\w'.
Python 3.8 rejects this, and it's confusing. This change fixes seven of these
regex strings found in depot_tools (through a regex search, natch). Most of
the fixes don't actually change the value of the strings, and this was
manually verified:
>>> '(/c(/.*/\+)?)?/(\d+)(/(\d+)?/?)?$' == r'(/c(/.*/\+)?)?/(\d+)(/(\d+)?/?)?$'
True
>>> '#\s*OWNERS_STATUS\s+=\s+(.+)$' == r'#\s*OWNERS_STATUS\s+=\s+(.+)$'
True
>>> 'COM\d' == r'COM\d'
True
>>> '^\s+Change-Id:\s*(\S+)$' == r'^\s+Change-Id:\s*(\S+)$'
True
>>> 'ETag:\s+([a-z0-9]{32})' == r'ETag:\s+([a-z0-9]{32})'
True
Two exceptions were the regex expressions in filter_demo_output.py and scm.py.
These were turned into raw strings despite this changing the value of the
string passed to re. This works because re supports the \x, \d, \w, \t, and
other escape sequences needed to make this work.
TL;DR - use raw strings for regex to avoid melting your brain. If bulk changing
regex strings to raw watch out for double-slashes.
Bug: 958138
Change-Id: Ic45264cfc63e8bae9cfcffe2cd88a57c2d3dcdae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1590534
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
The 'is' operator compares references, so a comparison with a string literal will always be false.
Found while using Python 3.8:
C:\Google\depot_tools/git_cl.py:1265: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if remote is not '.':
Change-Id: Ia77a1b350a3f0a2fe3c398e9a3b9aac8cfe38d4f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1591516
Auto-Submit: Raul Tambre <raul@tambre.ee>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Raul Tambre <raul@tambre.ee>
Regex strings need to be raw or else double-escape all the \ characters,
but Python 2.7 doesn't reliably enforce this, so bugs have crept in.
The buggy regex from create_installer_archive.py that this was copied
from was deleted a while ago. However this bad pattern is pervasive in
Chrome and depot_tools, as crudely found by searching for:
re\.[a-z]*\('.*\\[dsw\(\.]
An earlier instance of this was found when running some of our scripts
with Python 3.8.
Bug: 958138
Change-Id: If7ded5ae13f8cc36a5f6277c6ae0a2f54f88c3e4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1590191
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Collect git traces and prompt the user to upload them
when filling a bug.
Change-Id: Ic89fc848fdbfd497bd220dd54abf4ba35f40c39a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1555513
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
got_revision and got_revision_cp point to the commit that
Build.output.gitiles_commit should point to.
Therefore, derive output commit from these properties.
Derive output ref from got_revision_cp if it available.
Otherwise, try to read it from input commit
Bug: 940214
Change-Id: I108d782ad93c70114581830d7427b95d3efd9d98
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1588506
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Previously, gclient will exit if it found a tilde in any element from
PATH. This CL changes that to first check if the path is the actual
depot_tools directory.
Also check for '~', instead of '~/', in case the path looks like
'~user/path/to/depot_tools/.
Bug: 952865
Change-Id: Ied07444e44edb655b5c8837b5e51eeb0dcebba6d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1581419
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Henrique Ferreiro <hferreiro@igalia.com>
Knows about GCE Provider protos and some new fields in CQ protos. No API
changes.
R=tandrii@chromium.org
Change-Id: I23e131423884d6ac785810a24526c0a89e32c328
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1586375
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
The only upside of fetching everything directly from GoB
is that git cache would get just 1 .pack. However, git cache
already runs "git gc" before uploading cache to GS, so
this point is moot.
R=maruel
Bug: 943696
Change-Id: Ie8e77a81aa81489dae240b7c767c5842a12c6f19
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1585641
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Update metrics.README.md and add a metrics_version field.
Bug: None
Change-Id: I17d9032bd787af2a6fc49f80d59e05152d5223b7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1406256
Reviewed-by: Andy Perelson <ajp@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
The current owners systems scales badly on the number of files in a
patch, on the number of OWNERS files and on the number of "set noparent"
rules. This patch addresses all three of those.
If you have a patch with many (thousands) of files, you run into a lot
of repeated work because the parent directories are visited for every
file. By using a cache or remembering what directories have been
visited, a lot of work can be avoided.
If you have many OWNERS files or "set noparent" rules in scope of a
patch, owners and stop_looking lookups became slow. By splitting the
rules by longest glob-free directory path, you get much smaller lists
to process, which makes the code much faster.
(Test case with "all the files" went from >15 minutes to 30 seconds).
This saves about 2.5 second for git cl upload with a patch of
350 files spread over the tree.
The addition of many noparent rules in the Chromium tree might have
been the major reason this is needed, but there are also more files to
change, allowing larger patches.
Bug: 920591
Change-Id: If21178746ca9b88c2c07d265bd583b556d1734b7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1404172
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Auto-Submit: Daniel Bratell <bratell@opera.com>
Commit-Queue: Daniel Bratell <bratell@opera.com>
When NINJA_SUMMARIZE_BUILD=1 then autoninja prints a summary of build
performance at the end. This change makes it so that ninja prints more
detailed build statistics during the build. In particular, slow process
creation is a common but difficult to see bottleneck. This change makes
it so that if NINJA_SUMMARIZE_BUILD=1 then the number of running build
processes will be displayed.
Bug: 787983
Change-Id: Ic0907e23e7f762e23e4795059b37085e0cb8c4ea
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1582802
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
This change should hopefully enable the openscreen repo to use the
run_presubmit recipe.
Bug: openscreen:43
Change-Id: I672cddfeddd09ccc25ca851c0b613293191f17f9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1574189
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Bash and its descendants allow having $PATH entries containing literal tildes,
like:
...:~/depot_tools:...
If the user has their shell configured this way, their shell will be able to
invoke `fetch` and other depot_tools commands, but depot_tools will not in turn
be able to invoke subcommands via popen because popen does not have this
bash-specific behavior when not in shell mode.
This change has gclient detect this configuration problem and error out with a
descriptive error message. Not doing so leads to a puzzling condition where
gclient complains about being unable to find a program that (from the user's
perspective) is in their $PATH and invokable interactively.
It also has 'fetch' suppress its own failure stack trace if it runs gclient and
gclient fails; the 'fetch' stack trace pushes the messages from gclient (which
are actually diagnostic) off the screen.
Bug: chromium:952865
Change-Id: Ibba4d2fccee405aa68392ce141493f1de21ec018
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1573204
Commit-Queue: Elly Fong-Jones <ellyjones@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
A previous CL replaced "basestring" with "str", because basestring
does not exist in Python 3. However, this broke Python 2's ability
to interoperate with unicode strings. This CL introduces a workaround
(defining basestring to be equivalent to string, if it doesn't exist
already), and restores the references to basestring. This workaround
can be fixed when we're 100% on Python 3.
It also undoes some unnecessary and harder-to-read formatting changes.
Bug: 942522
Change-Id: I4a31ee46dc048134c2e4832b6c44ea00ce341899
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1572441
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
- It uses a different commit subject than recipe roller.
- Switch from looking up for magic strings to use a regexp instead. Document the
source of each string, to help keep up when the source diverge.
Tested manually.
Change-Id: I76fc6b6692508c615a0e76c4b6b403f8c513dd31
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1564938
Commit-Queue: Marc-Antoine Ruel <maruel@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Restore previous behavior and keep emitting a warning when an existing
gclient entry is not longer part of the client. Additionally, suggest
the usage of `gclient sync -D` to automatically remove them.
Change-Id: I62d4662ae4d0886d340230019419b68debffc5ba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1483031
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Henrique Ferreiro <hferreiro@igalia.com>
Oleh Prypin
Bruce Dawson
recipe-roller
Raul Tambre
Edward Lemur
John Budorick
Stephen Martinis
Nodir Turakulov
Henrique Ferreiro
Robert Iannucci
Karen Qian
Derek Schuff
Vadim Shtayura
Andrii Shyshkalov
Daniel Bratell
btolsch
Elly Fong-Jones
Aaron Gable
Marc-Antoine Ruel