`git cl split` claims to have built-in auto-resume capabilities, but
the nondeterminism in its splitting algorithm means they don't work
in practice. Now that we have the ability to load splittings from
files, we can ensure the user is using the same splitting as before,
and resume safely.
This CL does two things:
- It checks that any existing split branches match the splitting we're
about to upload, and complains if they don't match. This relies on
the fact that our branch names are unique and deterministic.
- It changes the auto-resume message to mention that --from-file is
required, and includes the relevant filename. This requires tracking
that filename from earlier in the program.
Bug: 389069356
Change-Id: Ic1d8964e96193ca93e05a9a39e286b84ffb61b06
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6280953
Commit-Queue: Devon Loehr <dloehr@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Add the following items to the allowlsted licenses based on the
requirement of crrev.com/c/6239492 :
- PngSuite
- Spencer-86
- unicode_org
Bug: 378472917
Change-Id: I6dfa79d612131eae154e5030ec22b787ebc57060
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6243133
Commit-Queue: Jordan Brown <rop@google.com>
Auto-Submit: Ramin Halavati <rhalavati@chromium.org>
Reviewed-by: Jordan Brown <rop@google.com>
To ease the UX of finding the branch name when there's a conflict to
resolve.
Change-Id: I34d16f2f7bd897a60e96b0391772b5d8985f3cb6
Bug: none
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6278647
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Paul Irish <paulirish@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
metrics_xml_format checks the given file path to ensure that
it's an XML under tools/metrics with the following steps.
1. convert the input file path as a real path.
2. check the real path starts with
GetPrimarySolutionPath() + "/tools/metrics"
This doesn't work as intended if GetPrimarySolutionPath() returns
a symblink.
This patch converts GetPrimarySolutionPath() as a realpath to ensure
it runs the check based on the realpath of the git checkout.
Bug: 396182402
Change-Id: I3207eba7ed52d09a01b38d65fcf7c314622d95cb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6279842
Auto-Submit: Scott Lee <ddoman@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
These files are testonly but can still be formatted by the
mojom formatter, so add to the matched extensions.
Bug: 397343888
Change-Id: I60352e8400e68426f262eb5601b76c17c0f0b912
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6279546
Reviewed-by: Scott Lee <ddoman@chromium.org>
Reviewed-by: Andy Perelson <ajp@google.com>
Commit-Queue: Alex Gough <ajgo@chromium.org>
This reverts commit d447f41294.
Reason for revert: "AttributeError: 'str' object has no attribute 'Extension'"
Original change's description:
> Add AffectedFile.Extension() in presubmit_support
>
> Add convenience method AffectedFile.Extension() for extracting file
> extension in a standard way. Add a test for this new functionality.
> Add a similar test for AffectedFile.UnixLocalPath().
>
> Bug: None
> Change-Id: If591e751fb2e4fb5355ad0b6f93f310667849d68
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6273341
> Commit-Queue: Anton Bershanskyi <bershanskyi@gmail.com>
> Reviewed-by: Dirk Pranke <dpranke@google.com>
> Reviewed-by: Gavin Mak <gavinmak@google.com>
Bug: None
Change-Id: Ic51b0942e3a3267194fb73af5e92c902b2098c4c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6279837
Auto-Submit: Gavin Mak <gavinmak@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Add convenience method AffectedFile.Extension() for extracting file
extension in a standard way. Add a test for this new functionality.
Add a similar test for AffectedFile.UnixLocalPath().
Bug: None
Change-Id: If591e751fb2e4fb5355ad0b6f93f310667849d68
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6273341
Commit-Queue: Anton Bershanskyi <bershanskyi@gmail.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
It turns out python's built-in hash function uses a random seed
that changes per-process, so branch names based on that hash
aren't actually deterministic. This changes the hashing algorithm
to use a stable hash so we can get consistent results.
Bug: 389069356
Change-Id: I1d3241b922005a7bff4d8621dc79dc4551bf264e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6258544
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Devon Loehr <dloehr@google.com>
Those warnings end up going into man pages, which is not desired. The
warning says that pipes are deprecated, and will be removed in py3.13.
R=yiwzhang@google.com
Bug: 396174247
Change-Id: Ieb7f36de944e8b7b5e68f78c87a61b04e663f9d1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6268243
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
During testing, I discovered that the try/catch block surrounding
most of SplitCl is broken: subprocess errors seem to return bytes
more often than strings. When I fixed that, however, I found that
the errors were much more comprehensible if we simply re-raised
the initial error and got all the information it contained; this
both gives us a stack trace and makes it very obvious an error
occurred.
Specifically, we also had the problem that stderr was sometimes
empty even if the process failed (because it wrote to stdout instead),
and if we just wrote the output then it wasn't clear that an error
happened until you actually read the output, as opposed to the
exception printing where the stack trace makes it obvious.
Since re-raising without additional work is pointless, I just removed
the block entirely.
Bug: 389069356
Change-Id: I1d1bec765469d0b70b463f7214f47a25316a37c3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6258516
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Devon Loehr <dloehr@google.com>
Tweak the code in ninja.py so it would only skip directories in $PATH
that are named "depot_tools". The existing detection logic skips
"foo_depot_tools". Also rename functions in the file to follow Python
Style and auto-format the file.
Change-Id: I631958768168f0d673b824c8e14783d2c6f65563
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6259139
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Lei Zhang <thestig@chromium.org>
See crbug.com/40264739 for many more details. But a prior attempt
to land this patch resulted in [1] which simply removed all
squashing behavior from `git rebase-update`. That broke several
people's workflows. So this patch attempts, again, to simply
add an opt-in to *not* squashing, for the folks (like me) who
do not want that behavior.
[1] https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6104282
Fixed: 40264739
Change-Id: I4a6aa8e53e854a7e601a0fb83f4f9fcb638d36ed
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6254099
Reviewed-by: Mason Freed <masonf@chromium.org>
Commit-Queue: Mason Freed <masonf@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mason Freed <masonf@chromium.org>
4 tests consistently error with "FileNotFoundError". Ensure that both
DEPS and gclient config use the same "file://" URI format so gclient
recognizes the repo as identical and doesn't move src/ to _bad_scm/.
Bug: 391455129
Change-Id: I507750754adc5e83b8a8ce853013a20f60102096
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6259135
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
This reverts commit 58625e82c6.
Reason for revert: Several comments on crbug.com/40264739 that this broke their workflow. I plan to revert this and put patchset #2 back up for review, with an opt-in instead of a change to the default behavior.
Original change's description:
> Remove "attempt to squash" behavior from `git rebase-update`
>
> In common workflows, this step only succeeds ~5% of the time, and
> in the other 95% of cases, simply adds ~30 seconds of wasted time
> to the execution time of `rebase-update`. Therefore, this CL removes
> the automated functionality, and leaves squashing as a manual option
> to the user.
>
> Fixed: 40264739
> Bug: 40390274
> Change-Id: Ib2a3ffe4b0e5d0b74323a2a0d34ab0e1b7fafb08
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6104282
> Auto-Submit: Mason Freed <masonf@chromium.org>
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Bug: 40390274
Change-Id: I91210a5a21b3751695553433389f346ba443bb1e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6254237
Auto-Submit: Mason Freed <masonf@chromium.org>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
This CL implements the proposal (from the design doc in the cited bug)
to allow `git cl split` to save the generated splitting to a
human-readable and editable text file. This allows users to save the
splitting in case something goes wrong and they need to retry, and also
lets them tweak a generated splitting if they get something mostly, but
not quite, usable.
It also allows users to edit the generated splitting interactively, like
they do for e.g. CL descriptions during `git cl upload`.
Bug: 389069356
Change-Id: I8ae21f2eb7c022ba181ae5af273ce09605b5acec
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6235727
Commit-Queue: Devon Loehr <dloehr@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Also adds an echo to win_tools.bat so a user will know when the
bootstrapping script is skipped after a CIPD install error.
This should hopefully help when troubleshooting.
Bug: b/360206460
Change-Id: Ic8bfcbddfe4681a7716ca9f3793d045e0de704cf
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6242904
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Anne Redulla <aredulla@google.com>
This CL changes the way `git cl split` generates branch names during
upload. Specifically, it attempts to derive a common directory for all
the files in each CL, and includes that dirname as part of the branch
name, in addition to a hash of the files in the CL. This makes it a bit
easier to guess what's in a branch, given its name.
Since the hash is still included, branch names should remain unique.
Finding a common directory should be deterministic, so we can still rely
on branches having the same name if the script is re-run with the same
splitting.
Bug: 389069356
Change-Id: I70490258755f13962f3db5c835619caa24176af9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6243377
Commit-Queue: Devon Loehr <dloehr@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
`git cl split` operates by creating a new git branch for each CL
it uploads. It names this branch based on the directory of the files
in the CL. It uses this convention when resuming an interrupted upload:
it skips branches that already exist.
However, this approach is brittle for several reasons:
- The splitting is nondeterministic, so there's no guarantee that CLs
with the same directories will be generated when re-run. Even if they
are, there's no guarantee they'll have exactly the same files.
- For CLs with multiple directories, it uses an arbitrary one, so it
may not choose the same name a second time
- It bakes in an assumption that every CL is defined by the directories
it draws files from.
This CL alleviates this issues by instead naming branches using the hash
of their contained files. This ensures that, with extremely high
probability, branches will have the same name if and only if they have
the same (sorted) list of files. Thus resumed uploads can safely skip
existing branches.
Bug: 389069356
Change-Id: I19bf7cd6ff8b0127f5906384f7f43b3e40ac8fee
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6236622
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Devon Loehr <dloehr@google.com>
This updates the presubmit error message when running on Cog to
include instructions on how to trigger presubmits. This also
standardizes the messaging across git cl presubmit/web/presubmit.
Bug: 394615879
Change-Id: Iaa8857268cd43219f7502eb34388034890a919dc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6240212
Reviewed-by: Andy Perelson <ajp@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
This adds AffectedFile.UnixLocalPath(), Change.UnixLocalPaths(), and
input_api.UnixLocalPaths() to the presubmit support APIs, so that we can
check paths in a platform-independent manner (i.e., not have to worry
about whether we're running on Windows and might need to handle
backslashes).
Change-Id: I1a424825384cb950ef2d4d3008c9c6e60028cf07
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6236404
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Dirk Pranke <dpranke@google.com>
This is a preparatory CL which refactors some internal logic to make
writing future CLs easier. In particular, it changes from treating a
CL as "a list of files with the same reviewer" to treating them as a
triple of (reviewers, files, directories). It does so using a dataclass
to make it easy to work with.
It also adds an EmitWarning convenience function, which will be more
heavily used in future CLs.
Bug: 389069356
Change-Id: I842d59ba69f5db5e3745fa39832794cdc1eb222e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6190430
Commit-Queue: Devon Loehr <dloehr@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Introduces a new git cl command `squash-closed`. Similar to archive,
this will operate on all closed branches. Unlike archive, rather than
removing these branches and potentially leaving their downstreams in a
wedged state, this will squash the branch and appropriately reparent the
downstream. This makes this a good function to run in preparation of a
`git rebase-update`, especially if there are chained git branches, as
this will then shrink any merged branches to a single commit which
should cleanly apply and drop off.
Bug: 40264739
Change-Id: I1836f944d43f5f3dcebbebf06437bef890da96ca
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6230362
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Alexander Cooper <alcooper@chromium.org>
Auto-Submit: Alexander Cooper <alcooper@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Devon Loehr
recipe-roller
Alexander Cooper
Ramin Halavati
Paul Irish
Scott Lee
Alex Gough
Jordan
Chan Li
Gavin Mak
Anton Bershanskyi
Josip Sokcevic
Andrew Grieve
Anne Redulla
Fumitoshi Ukai
Josip Sokcevic
Lei Zhang
Mason Freed
Jiacheng Guo
Dirk Pranke
KS Yim