This makes it much faster to get a diff for a set of multiple files.
Bug: b/323243527
Change-Id: I6c54a4fa8782d4a0bbb187b8bed4fbd52be47344
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5384647
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
This CL adds a "structured" concept to the parser. In a structured
field, the parser will proactively look for field-like patterns to
start a new field (even if they aren't known fields).
This mitigates the issue when an unknown field immediately
follows a multi-line text field, such as:
URL: https://example.com
UnknownField: abc
And URL field value parses to
"https://example.com<newline>UnknownField:abc".
Bug: b/324149233
Change-Id: I54807bd7b242fc14c679483453ade83f8fd20225
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5379679
Reviewed-by: Anne Redulla <aredulla@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
This is a reland of commit 4d2728b03e
Added default values for host and project in case input.gerrit does not
exist.
Original change's description:
> Include host and project in code-owners not enabled error.
>
> Bug: 329231688
> Change-Id: I36bb24f167854f40718ec49559d9aaa640fa1a44
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5384365
> Auto-Submit: Joanna Wang <jojwang@chromium.org>
> Commit-Queue: Joanna Wang <jojwang@chromium.org>
> Reviewed-by: Scott Lee <ddoman@chromium.org>
> Commit-Queue: Scott Lee <ddoman@chromium.org>
Bug: 329231688
Change-Id: If65e362d0aba97d804a75c078452bab8152ae224
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5384366
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Scott Lee <ddoman@chromium.org>
This reverts commit 4d2728b03e.
Reason for revert: Won't work if gerrit=None
Original change's description:
> Include host and project in code-owners not enabled error.
>
> Bug: 329231688
> Change-Id: I36bb24f167854f40718ec49559d9aaa640fa1a44
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5384365
> Auto-Submit: Joanna Wang <jojwang@chromium.org>
> Commit-Queue: Joanna Wang <jojwang@chromium.org>
> Reviewed-by: Scott Lee <ddoman@chromium.org>
> Commit-Queue: Scott Lee <ddoman@chromium.org>
Bug: 329231688
Change-Id: I49d23611b62f1979578ae52861a2007a1d34778e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5381788
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Explicitly sorts the discovered metadata file, and defines
ordering of validation messages, so the tools will output
will be self-consistent.
Change-Id: I9b263a16b151c014e5950638f066376469c701df
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5379678
Commit-Queue: Jiewei Qian <qjw@chromium.org>
Reviewed-by: Anne Redulla <aredulla@google.com>
Bug: 329231688
Change-Id: I36bb24f167854f40718ec49559d9aaa640fa1a44
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5384365
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Scott Lee <ddoman@chromium.org>
Occasionally on windows there seems to be a racing problem where rmtree
tries to delete something that no longer exists. This seems to happen
when a build is cancelled/terminated while in-progress, so racing
actions in reproxy might have deleted the temporary file while autoninja
is cleaning up stuff. In any case, missing a file in this case isn't
something to fail over.
Bug: b/329738642
Change-Id: I86240e31093c6415707878277b01db5931b175ba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5376532
Reviewed-by: Ben Segall <bentekkie@google.com>
Commit-Queue: Michael Savigny <msavigny@google.com>
os.walk is more efficient than the current handwritten traversal.
Measured the time to scan reduced from 30s+ to 8s on p920 on
chromium/src.
`followlinks=True` is set to preserve behavior that os.path.isdir
returns True for symlink to directories, and the current traversal code
will descend into those.
Change-Id: I941eec9105a46d6538ca484fbb5249a75888e38a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5379945
Reviewed-by: Anne Redulla <aredulla@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
If is_first_class_gcs file exists with contents '1'
- GCS hooks will download content and then overwrite the file with '0'
- First class GCS deps will download as necessary by checking the hash
file.
If there is no is_first_class_gcs file or its content is '0' for a GCS hook:
- Hook will run normally and, if content is downloaded, an is_first_class_gcs
file will be written with '0'
If there is no is_first_class_gcs file or its content is '0' for a first
class gcs dep:
- Content will be downloaded and is_first_class_gcs will be set to '1'
Verified locally with migrating and unmigrating a GCS hook.
TODO: update other GCS hook scripts in chromium/src to check migration
file.
Bug: b/324418194
Change-Id: Ida8541cb70839b86e99115bcabfc38bc87e40136
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5368250
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
This flag allows a diff to be generated based on an upstream commit
for provided files and for checks to be run with on that diff.
Bug: b/323243527
Change-Id: Ia3f4ec50b15eabb90d391b9bc795fc4b35a35a08
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5376402
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Create a script to create a unified git diff of a local workspace
against a remote Gerrit commit. This will be used to create a diff file
for running presubmits.
Bug: b/323243527
Change-Id: Ia0d287624162dbfe5231a5653c9a962a6c85c8e1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5374833
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
GCS deps will require a size_bytes field of type `int`. If gclient
sees that the actual byte size is different than the noted size, it
will raise an exception.
Bug: b/328065441
Change-Id: If61496eae39cd372966d7e64ba4a84759708c60b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5367540
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
I initially didn't want to make this code Google-specific, however I now
believe that it's better for everyone to simplify it like this:
- It's much less code to just check for the path prefix. We do this
already in gclient.py, too, so the approach is proven to work.
- The call to findmnt introduces a 15ms overhead that affects everyone,
even users who will never have their workspace on Cog.
- External folks who run their own infrastructure likely have their own
fork of depot_tools anyway and can patch this as needed.
Bug: b/324547324
Change-Id: Ic00d417c280ac71529d106cfb2bcc844dfd44bd3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5358773
Commit-Queue: Philipp Wollermann <philwo@chromium.org>
Auto-Submit: Philipp Wollermann <philwo@chromium.org>
Reviewed-by: Junji Watanabe <jwata@google.com>
This doesn't catch all the variants of --help that getopt allows but it
does catch --help, which is probably the most common.
Bug: 329317224
Change-Id: I2dde63df84bd2c2ade5f9d2c1ab10c3dc13fba70
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5370829
Auto-Submit: David Baron <dbaron@chromium.org>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
google-java-format will only change the lines given by --lines, which
means that:
a) Unused imports are not removed (assuming they were existing imports)
b) Excessive blank lines between methods are not removed
To address this, run google-java-format a second time with
--fix-imports-only, include deleted lines in ranges, and expand ranges
by 2 lines on each side.
Bug: 1462204
Change-Id: Ia12af6def28d9fe11e8315a951d45908f7cee725
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5362198
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Andrew Grieve <agrieve@chromium.org>
Notes that --offline is for disabling remote execution in general not just for goma.
Bug: b/329317224
Change-Id: I5538c9fac6d5161e98e1d156e61c512c6a1e8496
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5370094
Auto-Submit: Michael Savigny <msavigny@google.com>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
When an 'output_file' arg is included, the downloaded object from GCS
will be downloaded in that path.
Example:
```
'src/buildtools/linux': {
'bucket': 'bucket123',
'object_name': 'clang-format-version123',
'dep_type': 'gcs',
'sha256sum': 'abcd123',
'output_file': 'clang-format',
}
```
The GCS object will be downloaded at src/buildtools/linux/clang-format
Bug: b/324418194
Change-Id: I1049abeb09a1027c5477d955e50611d43015d0a4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5353387
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
The check is intended to ensure paths don't traverse outside
the extracted directory. However, the check was too strict: it
banned all links, even relative links that still point inside the
target directory. This CL relaxes the requirement to allow valid
links. This is required to allow library symlinks for the
instrumented libraries. Examples:
libpcre.so -> libpcre.so.3.13.3
libpcre.so.3 -> libpcre.so.3.13.3
libpixman-1.so -> libpixman-1.so.0.38.4
libpixman-1.so.0 -> libpixman-1.so.0.38.4
libpng16.so -> libpng16.so.16.37.0
libpng16.so.16 -> libpng16.so.16.37.0
Bug: 320564950
Change-Id: I2aae18b86b1f1cc3d73a1b80c06d757af782f700
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5369132
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Auto-Submit: Thomas Anderson <thomasanderson@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
A recent change added --diff support for Java files in git cl format,
but unfortunately it broke the behavior when both --dry-run and --diff
are passed, and made git cl format indicate that all changed files need
to be reformatted.
Fix so that --diff overrides --dry-run when running google-java-format,
as passing --dry-run makes the formatter only output file names which
then confused the diff creation code. This makes behavior for Java in
line with C++ where --dry-run doesn't change much if --diff is used.
Change-Id: I7f8eecb13064a0046935489e6c5cf3af566f7f56
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5357818
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Tomasz Śniatowski <tomasz.sniatowski@xperi.com>
Changes in this release:
b90a6a746293e85cf68ee1b43643bf1fca1908a8 vpython: use tmp dir if can't determine cache dir
20ed5814e481a4adfe86b0c128f382175e769af3 cipkg: copy action shouldn't use path when calculating derivation hash
55717d93b38ae907d65de9b13630e13d9913d284 vpython: always use absolute vpython root dir
Bug: 325535679
Change-Id: I6772d6225729397443a77cff5bdca6b2d2225345
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5352387
Reviewed-by: Chenlin Fan <fancl@chromium.org>
Commit-Queue: Brian Ryner <bryner@google.com>
In source, line-crossing string fragments are joined with no spaces in
between. This CL trivially adds spaces s.t. the warning doesn't get
printed like:
```
won'tbreak anything...to cancelyour jobs
```
Change-Id: I17b4a6c0d22fa63ee4ea66d5b32f7fc3221bc2be
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5352388
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Kalvin Lee <kdlee@chromium.org>
We use vpython3 in autoninja.bat now.
Current difference of python3 and vpython3 for autoninja.py invocation
is like belows and I think several hundred milliseconds is acceptable
increase for the build.
gLinux:
* vpython3: 240.0ms
* python3: 99.1ms
https://pastebin.com/LK8kUAHn
gWindows:
* vpython3: 912.7ms
* python3: 611.8ms
https://pastebin.com/Xqs77ipv
MacBook Pro:
* vpython3: 386.2ms
* python3: 280.1ms
https://pastebin.com/eyXEpM7Q
Bug: 327625559
Change-Id: Iff52be12cb755b3ecdab6663a0038905cf178d53
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5348020
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Google Java Format does not support `--diff` parameter, but we can simulate it by manually diffing the output of the tool with the current
file on disk.
All git cl-triggered format tools should support --diff output, because there're situations where a change in a file might be required, but it's unclear what exactly should be changed, because a developer might not have access to the host OS which only runs the check.
We at Brave want to run `git cl format --diff` as a part of CI run when a format check fails.
Change-Id: I2feee60219a54ad44b93bc10f5488a9c05715a4a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5351216
Reviewed-by: Andrew Grieve <agrieve@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Aleksey Khoroshilov <akhoroshilov@brave.com>
The `-Y` flag was added to `df` only in the latest macOS version, so we
should guard its usage with a version check.
Change-Id: Ifafe47de5f9e86ce6a4605a1adb26994db5f33d9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5345461
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Philipp Wollermann <philwo@chromium.org>
Auto-Submit: Philipp Wollermann <philwo@chromium.org>
Gavin Mak
Jiewei Qian
Josip Sokcevic
recipe-roller
Joanna Wang
Joanna Wang
Michael Savigny
Robert Iannucci
Josh Berenhaus
Stephanie Kim
Philipp Wollermann
L. David Baron
Andrew Grieve
Tom Anderson
Tomasz Śniatowski
Brian Ryner
Xianzhu Wang
Kalvin Lee
Takuto Ikuta
Aleksey Khoroshilov
Michael Ershov
Ben Segall
mlcui