Instead of using build/toolchain/use_reclient_value.py in
both autoninja and gn, use build/toolchain/remoteexec_defaults.gni
to set default value for use_reclient (on siso or on ninja).
Bug: 397994249
Change-Id: Ia4302fe8794218d485e52ca38f8523453814a6b9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6301378
Reviewed-by: Andrew Grieve <agrieve@chromium.org>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
This will later be used in chromium PRESUBMIT to ensure newly added DEPS
hook MUST be reviewed by a dedicated reviewer.
Bug: 396736534
Change-Id: I3859814c6316d4e576d12114277671ed5db80e33
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6284565
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
chromium[1] and cros[2] have both moved to 3.11 which is the version
depot_tools use by default right now. Therefore, we should stop running
tests using python 3.8 to ensure compatibility and start using features
introduced after python 3.8
[1]: https://issues.chromium.org/40942322
[2]: http://b/318531403
Change-Id: I2bac00421a46793717387cec1898035b10b13b9e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6306285
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
When registering the build with the build server, pass in the output
directory.
Change-Id: Ie6ed0bf5d73926fb163085067aafcd350063ee5a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6249305
Auto-Submit: Mohamed Heikal <mheikal@chromium.org>
Reviewed-by: Philipp Wollermann <philwo@google.com>
Commit-Queue: Mohamed Heikal <mheikal@chromium.org>
chromium now doesn't use `use_siso` value for build graph,
so no need to use use_siso_default.py in gn context.
use_siso value is only used to determine autoninja use
siso or ninja, so put use_siso default logic in autoninja.py.
Will remove build/toolchain/use_siso_default.py in chromium
Bug: 397994249
Change-Id: Idb31c4e75f0a1c3bc4a97089494ec16be9879a49
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6301272
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
This CL adds a --reviewers flag, which allows users to specify that CLs
should all be sent to the same (possibly empty) set of reviewers. This
can be useful for LSCs where OO+1 is available.
Bug: 389069356, 40269201
Change-Id: Ic6202b280bb676f75a71c4ccf0c41b32483d5c6e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6298664
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Devon Loehr <dloehr@google.com>
This is because the linux-presubmit ci job does not run with all
dependencies checked out. See bug for details.
Bug: 398970704
Change-Id: Ia562cc6de7e586f947ccc9d351e9fc5feafa9f22
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6300962
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Jordan Brown <rop@google.com>
`git cl split` allows users to parameterize their CL descriptions
by including the string $directory, which is replaced by the list of
directories covered by each generated CL. However, as we add more ways
of generating splittings, a list of directories may not always be the
most appropriate description. For example, the --from-file option lets
users write arbitrary description strings.
This CL renames the $directory string to $description, to reflect this
new generality, and refactors all places in the code that expected a
directory list instead of a string. Since users may beused to the
current version, it does not remove $directory, but instead notes that
it is deprecated and emits a warning when it is used. After some time,
we can remove it entirely.
Bug: 389069356
Change-Id: If8c947fdcbbb4897675b015a377cf21123e51467
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6299688
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Devon Loehr <dloehr@google.com>
Currently, the dry run setting causes the script to print out extremely
detailed information about the splitting it generated. While this is
sometimes useful, often users only care about the overall shape of the
splitting. This CL adds a flag which lets them see the same brief
summary that's printed during full runs.
Bug: 389069356
Change-Id: I473740e50f382e63b1289101f3253ff8ae9e6bbe
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6298096
Commit-Queue: Devon Loehr <dloehr@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Python has started warning about invalid escape sequences in strings.
This fixes the warning by escaping the backslash in question. It seems
Python was already (correctly) printing the literal backslash despite
the warning, so no behavioral change.
Bug: None
Change-Id: I2d5d8d56b989d9a1bb87aba69af0ebac0209a7a0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6299490
Commit-Queue: Devon Loehr <dloehr@google.com>
Reviewed-by: Brian Ryner <bryner@google.com>
This CL replaces the builtin print function with an alias `Emit`.
This is symmetric with `EmitWarning`, and primarily serves to allow
easier mocking during tests and debugging. No functional change.
Bug: 389069356
Change-Id: Ib2f5a7648458b6b1181606a3dccd9829542c5521
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6299489
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Devon Loehr <dloehr@google.com>
settings.GetFormatFullByDefault() looks up a git config to find if
format-full-by-default is set.
RunClangFormatDiff() and RunGoogleJavaFormat() uses the function to
determine whether it should format a given document full or not, and
it makes it difficult to make the wrapper functions git-op free.
To remove git dependencies from the Run functions, this CL simply sets
opts.full if the default is set.
Note that this CL makes a behaviour change to the Python Runner.
RunYapf() doesn't honor settings.GetFormatFullByDefault().
It just looks at opts.full only to determine how to format the file.
If this CL is landed, RunYapf() will take account of the git config to
determine whether it should format the full document or a part.
Bug: 386840832
Change-Id: Ibd7210e20897d354bd3eddf84927cf22105b7255
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6256552
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Scott Lee <ddoman@chromium.org>
Ideally we would track these separately but practically they fall into
the same category as the restricted license. It is on the OWNERS and the
reviewers to make sure that they comply with the terms of a Patent file
or any license. Currently we are just generating presubmit warnings that
are being ignored.
This will also enable future tooling that utilises this list for
auditing restricted licenses to surface Patent files.
Bug: 381146326
Change-Id: I0f091bef9649d3a9f7b03940c8634e56bee9541f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6290872
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Jordan Brown <rop@google.com>
`git cl split` claims to have built-in auto-resume capabilities, but
the nondeterminism in its splitting algorithm means they don't work
in practice. Now that we have the ability to load splittings from
files, we can ensure the user is using the same splitting as before,
and resume safely.
This CL does two things:
- It checks that any existing split branches match the splitting we're
about to upload, and complains if they don't match. This relies on
the fact that our branch names are unique and deterministic.
- It changes the auto-resume message to mention that --from-file is
required, and includes the relevant filename. This requires tracking
that filename from earlier in the program.
Bug: 389069356
Change-Id: Ic1d8964e96193ca93e05a9a39e286b84ffb61b06
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6280953
Commit-Queue: Devon Loehr <dloehr@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Add the following items to the allowlsted licenses based on the
requirement of crrev.com/c/6239492 :
- PngSuite
- Spencer-86
- unicode_org
Bug: 378472917
Change-Id: I6dfa79d612131eae154e5030ec22b787ebc57060
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6243133
Commit-Queue: Jordan Brown <rop@google.com>
Auto-Submit: Ramin Halavati <rhalavati@chromium.org>
Reviewed-by: Jordan Brown <rop@google.com>
To ease the UX of finding the branch name when there's a conflict to
resolve.
Change-Id: I34d16f2f7bd897a60e96b0391772b5d8985f3cb6
Bug: none
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6278647
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Paul Irish <paulirish@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
metrics_xml_format checks the given file path to ensure that
it's an XML under tools/metrics with the following steps.
1. convert the input file path as a real path.
2. check the real path starts with
GetPrimarySolutionPath() + "/tools/metrics"
This doesn't work as intended if GetPrimarySolutionPath() returns
a symblink.
This patch converts GetPrimarySolutionPath() as a realpath to ensure
it runs the check based on the realpath of the git checkout.
Bug: 396182402
Change-Id: I3207eba7ed52d09a01b38d65fcf7c314622d95cb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6279842
Auto-Submit: Scott Lee <ddoman@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
These files are testonly but can still be formatted by the
mojom formatter, so add to the matched extensions.
Bug: 397343888
Change-Id: I60352e8400e68426f262eb5601b76c17c0f0b912
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6279546
Reviewed-by: Scott Lee <ddoman@chromium.org>
Reviewed-by: Andy Perelson <ajp@google.com>
Commit-Queue: Alex Gough <ajgo@chromium.org>
This reverts commit d447f41294.
Reason for revert: "AttributeError: 'str' object has no attribute 'Extension'"
Original change's description:
> Add AffectedFile.Extension() in presubmit_support
>
> Add convenience method AffectedFile.Extension() for extracting file
> extension in a standard way. Add a test for this new functionality.
> Add a similar test for AffectedFile.UnixLocalPath().
>
> Bug: None
> Change-Id: If591e751fb2e4fb5355ad0b6f93f310667849d68
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6273341
> Commit-Queue: Anton Bershanskyi <bershanskyi@gmail.com>
> Reviewed-by: Dirk Pranke <dpranke@google.com>
> Reviewed-by: Gavin Mak <gavinmak@google.com>
Bug: None
Change-Id: Ic51b0942e3a3267194fb73af5e92c902b2098c4c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6279837
Auto-Submit: Gavin Mak <gavinmak@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Add convenience method AffectedFile.Extension() for extracting file
extension in a standard way. Add a test for this new functionality.
Add a similar test for AffectedFile.UnixLocalPath().
Bug: None
Change-Id: If591e751fb2e4fb5355ad0b6f93f310667849d68
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6273341
Commit-Queue: Anton Bershanskyi <bershanskyi@gmail.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
It turns out python's built-in hash function uses a random seed
that changes per-process, so branch names based on that hash
aren't actually deterministic. This changes the hashing algorithm
to use a stable hash so we can get consistent results.
Bug: 389069356
Change-Id: I1d3241b922005a7bff4d8621dc79dc4551bf264e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6258544
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Devon Loehr <dloehr@google.com>
Those warnings end up going into man pages, which is not desired. The
warning says that pipes are deprecated, and will be removed in py3.13.
R=yiwzhang@google.com
Bug: 396174247
Change-Id: Ieb7f36de944e8b7b5e68f78c87a61b04e663f9d1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6268243
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
During testing, I discovered that the try/catch block surrounding
most of SplitCl is broken: subprocess errors seem to return bytes
more often than strings. When I fixed that, however, I found that
the errors were much more comprehensible if we simply re-raised
the initial error and got all the information it contained; this
both gives us a stack trace and makes it very obvious an error
occurred.
Specifically, we also had the problem that stderr was sometimes
empty even if the process failed (because it wrote to stdout instead),
and if we just wrote the output then it wasn't clear that an error
happened until you actually read the output, as opposed to the
exception printing where the stack trace makes it obvious.
Since re-raising without additional work is pointless, I just removed
the block entirely.
Bug: 389069356
Change-Id: I1d1bec765469d0b70b463f7214f47a25316a37c3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6258516
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Devon Loehr <dloehr@google.com>
Tweak the code in ninja.py so it would only skip directories in $PATH
that are named "depot_tools". The existing detection logic skips
"foo_depot_tools". Also rename functions in the file to follow Python
Style and auto-format the file.
Change-Id: I631958768168f0d673b824c8e14783d2c6f65563
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6259139
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Lei Zhang <thestig@chromium.org>
See crbug.com/40264739 for many more details. But a prior attempt
to land this patch resulted in [1] which simply removed all
squashing behavior from `git rebase-update`. That broke several
people's workflows. So this patch attempts, again, to simply
add an opt-in to *not* squashing, for the folks (like me) who
do not want that behavior.
[1] https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6104282
Fixed: 40264739
Change-Id: I4a6aa8e53e854a7e601a0fb83f4f9fcb638d36ed
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6254099
Reviewed-by: Mason Freed <masonf@chromium.org>
Commit-Queue: Mason Freed <masonf@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mason Freed <masonf@chromium.org>
4 tests consistently error with "FileNotFoundError". Ensure that both
DEPS and gclient config use the same "file://" URI format so gclient
recognizes the repo as identical and doesn't move src/ to _bad_scm/.
Bug: 391455129
Change-Id: I507750754adc5e83b8a8ce853013a20f60102096
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6259135
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
This reverts commit 58625e82c6.
Reason for revert: Several comments on crbug.com/40264739 that this broke their workflow. I plan to revert this and put patchset #2 back up for review, with an opt-in instead of a change to the default behavior.
Original change's description:
> Remove "attempt to squash" behavior from `git rebase-update`
>
> In common workflows, this step only succeeds ~5% of the time, and
> in the other 95% of cases, simply adds ~30 seconds of wasted time
> to the execution time of `rebase-update`. Therefore, this CL removes
> the automated functionality, and leaves squashing as a manual option
> to the user.
>
> Fixed: 40264739
> Bug: 40390274
> Change-Id: Ib2a3ffe4b0e5d0b74323a2a0d34ab0e1b7fafb08
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6104282
> Auto-Submit: Mason Freed <masonf@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Bug: 40390274
Change-Id: I91210a5a21b3751695553433389f346ba443bb1e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6254237
Auto-Submit: Mason Freed <masonf@chromium.org>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
This CL implements the proposal (from the design doc in the cited bug)
to allow `git cl split` to save the generated splitting to a
human-readable and editable text file. This allows users to save the
splitting in case something goes wrong and they need to retry, and also
lets them tweak a generated splitting if they get something mostly, but
not quite, usable.
It also allows users to edit the generated splitting interactively, like
they do for e.g. CL descriptions during `git cl upload`.
Bug: 389069356
Change-Id: I8ae21f2eb7c022ba181ae5af273ce09605b5acec
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6235727
Commit-Queue: Devon Loehr <dloehr@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Also adds an echo to win_tools.bat so a user will know when the
bootstrapping script is skipped after a CIPD install error.
This should hopefully help when troubleshooting.
Bug: b/360206460
Change-Id: Ic8bfcbddfe4681a7716ca9f3793d045e0de704cf
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6242904
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Anne Redulla <aredulla@google.com>
Allen Li
Peter Wen
Fumitoshi Ukai
Yiwei Zhang
Mohamed Heikal
Tomasz Śniatowski
Jordan Brown
Devon Loehr
Scott Lee
recipe-roller
Alexander Cooper
Ramin Halavati
Paul Irish
Alex Gough
Chan Li
Gavin Mak
Anton Bershanskyi
Josip Sokcevic
Andrew Grieve
Anne Redulla
Josip Sokcevic
Lei Zhang
Mason Freed