Remove unused functions, and use scm.GIT where possible to reduce
duplication.
Change-Id: I24f05e7b3ae04743e97b6665ee668f44d6acc294
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2116938
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
gclient_utils.AskForData will use raw_input on Python 2 and input on Python 3.
Bug: 1063976
Change-Id: I93c059c4e4454d01787716b5a0a2641ad5253f53
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2116370
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
gclient executes `git rev-parse <ref>` to check if <ref>
already exists in the local clone, which succeeds when
<ref> is a git revision hash, even if it doesn't exist.
We execute `git rev-parse <ref>^{commit}` instead, which only
succeeds when <ref> resolves to a valid commit in the local
repo.
Change-Id: Ia1cdf7162c4c82a7e4214c6af375c1a9522d7b07
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2095745
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Also add a better warning if 'gclient sync -D' fails due to a modified file.
I've tested this code via deleting a DEP and running gclient sync -D.
Bug: 981149
Change-Id: I97035ac238d163ccb1684c3ee423c223ed0f6299
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1891830
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Anthony Polito <apolito@google.com>
This changes the behaviour when applying a patch to not fetch tags
along with the patch being applied. The fetch to apply a patch only
needs the commits that will be cherry picked, it does not need tags.
Fetching tags creates slow bottlenecks if a repo such as chromium has
10k+ tags. Using --no-tags prevents the server from sending the list
of all tags and the client from processing them.
Bug: 1019824
Change-Id: I756beb8c2b0c7ec3c0fc48b7431d0cf8c0bbbbd3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1894393
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Erik Chen <erikchen@chromium.org>
Commit-Queue: danakj <danakj@chromium.org>
This makes no-rebase-patch-ref correctly populate `file_list` and work with
`git diff --cached` (i.e. they will show only the files affected by the
patchset).
Previously gclient would `reset --soft` to master. Because we didn't rebase
on top of master with `no-rebase-patch-ref`, it has roughly nothing to do
with the patchref we just checked out.
R=gbeaty@chromium.org, ltina@google.com, tandrii@chromium.org
Context:
Tricium recipes sometimes need to use no-rebase-patch-ref in order to get
accurate linenumbers in the diff (i.e. so that the tricium comments will
actually line up correctly with the patchset the user uploaded); Tricium
recipes also need to accurately get the diff file list in this mode.
Change-Id: I5f3c95cd4958cf407a83b96c238b8c55c452ac81
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1895041
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
Older .gclient files might have explicit u'' strings in their
specs; as part of upgrading code to Python3 compatibility,
we apparently recently introduced a bug in the handling of this
that might cause them to fail on Windows.
This CL fixes that issue by ensuring that the GIT_DIR env var
is always set to a regular string, regardless of whether
the solution name was specified to be a regular string
or a unicode string.
Bug: 1016599
Change-Id: I295d220559eac20a2ced876672d5ccdb787a5338
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1874704
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
If the .git/ subdir exists and mostly looks OK, but in reality git
doesn't accept it, gclient will blindly run git in the subdir and
think it's operating on the local .git/. Kind of like:
$ cd src/third_party/emoji-segmenter/src/
$ rm -rf .git && mkdir .git
# This works on the top level src/.git/.
$ git <operation>
Instead of relying on this automatic git behavior, use GIT_DIR by
default so we force git to only look at that specific checkout. If
it's broken, git will fail on us (good) instead of silently falling
back to the parent (bad).
$ cd src/third_party/emoji-segmenter/src/
$ rm -rf .git && mkdir .git
$ GIT_DIR=$PWD/.git git <operation>
<this now hard fails>
Bug: 1000825
Test: `gclient sync -j1` with a broken src/third_party/emoji-segmenter/src/.git/ caused the sync to halt
Change-Id: I777d67cd5daa354b32c424414d93a61b3b600f54
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1785957
Commit-Queue: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
I know that sometimes imports can have side-effects,
so unused imports shouldn't always be removed, but these
ones look like they could be.
Change-Id: Iea9f82afa99b0ea35f29a28f20ce0493b579cfee
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1819860
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
- Merge CheckCallAndFilter[AndHeader]
- print_stdout will cause command output to be redirected to sys.stdout.
- Made compatible with Python 3.
Bug: 984182
Change-Id: Ida30e295b872c8c1a1474a376a90aecea924f307
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1727404
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
This avoids the need to do the decoding everywhere separately.
Also a small fix for filter() no longer returning a list on Py3.
The scripts still work with Python 2.
There are no intended behaviour changes.
Bug: 942522
Change-Id: Id0f5153011b2ef1b64394359087864cd9434e45e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1595685
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
Auto-Submit: Raul Tambre <raul@tambre.ee>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
A previous CL replaced "basestring" with "str", because basestring
does not exist in Python 3. However, this broke Python 2's ability
to interoperate with unicode strings. This CL introduces a workaround
(defining basestring to be equivalent to string, if it doesn't exist
already), and restores the references to basestring. This workaround
can be fixed when we're 100% on Python 3.
It also undoes some unnecessary and harder-to-read formatting changes.
Bug: 942522
Change-Id: I4a31ee46dc048134c2e4832b6c44ea00ce341899
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1572441
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Restore previous behavior and keep emitting a warning when an existing
gclient entry is not longer part of the client. Additionally, suggest
the usage of `gclient sync -D` to automatically remove them.
Change-Id: I62d4662ae4d0886d340230019419b68debffc5ba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1483031
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Henrique Ferreiro <hferreiro@igalia.com>
This enables gclient sync and gclient runhooks to run, barring hook script failures.
git cl upload also now works.
The scripts still work with Python 2.
There are no intended behaviour changes.
Bug: 942522
Change-Id: I2ac587b5f803ba7f5bb5e412337ce049f4b1a741
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1524583
Commit-Queue: Raul Tambre <raul@tambre.ee>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
The scripts still work with Python 2.
There are no intended behaviour changes.
Bug: 939847
Change-Id: Icada60c5b2cf351d62aead26b7364fcef2c2a3e5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1524486
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Raul Tambre <raul@tambre.ee>
If we already have the correct commit, it is not necessary to fetch. Any
fetches can takes minutes due to gerrit problems, and often fail
completely, so it is helpful to avoid them completely whenever we can.
Reland now that skia autoroller does its own fetch.
Bug: 938627
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1504102
Commit-Queue: Michael Spang <spang@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Change-Id: I5a6877125c682ff0d98bd6d2ffe495245f74c513
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1512024
Auto-Submit: Michael Spang <spang@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
This reverts commit 61a1b5f310.
Reason for revert:
It's blocking skia autoroller
https://skia-review.googlesource.com/c/buildbot/+/198700
Original change's description:
> Avoid unnecessary invocations of "git fetch"
>
> If we already have the correct commit, it is not necessary to fetch. Any
> fetches can takes minutes due to gerrit problems, and often fail
> completely, so it is helpful to avoid them completely whenever we can.
>
> Bug: 938627
>
> Change-Id: I54494cad9b9aa0090f51674b6b9f56bb3e842f12
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1504102
> Commit-Queue: Michael Spang <spang@chromium.org>
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
TBR=spang@chromium.org,agable@chromium.org,ehmaldonado@chromium.org
Change-Id: If8bd26d74bf84ff0ca69378dbfe6b9369b9988d3
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 938627
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1508980
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
If we're asked to sync a repo to branch-heads or tags,
make sure we fetch refs/branch-heads/* and refs/tags/* even if
--with-branch-heads or --with-tags was not passed.
Bug: 938160
Change-Id: I04de8e7762c83322692d9551d057ec803fd2ac9c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1506420
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
If we already have the correct commit, it is not necessary to fetch. Any
fetches can takes minutes due to gerrit problems, and often fail
completely, so it is helpful to avoid them completely whenever we can.
Bug: 938627
Change-Id: I54494cad9b9aa0090f51674b6b9f56bb3e842f12
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1504102
Commit-Queue: Michael Spang <spang@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Also add shortcuts, so we don't have to look through all the branches
if the patch is based on a common branch (i.e. master, infra/config, lkgr)
Bug: 870279
Change-Id: I625a8481dccac9a475b096b926e6fab7efe676b0
Reviewed-on: https://chromium-review.googlesource.com/1161094
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Abort any cherry-picks before applying the patch, so that if the bots are in a
bad state, we don't fail.
Original change's description:
> Reland "gclient_scm: Use cherry-picking instead of rebasing."
>
> The failures were caused by:
> 1 - When one change (call it #2) has been uploaded on top of another (#1),
> and (#1) has already landed, git cherry-pick complains that the range
> '<merge-base>..<change #2>' contains empty commits, since the contents
> of (#1) are already present in the tree.
> 2 - We did not abort the cherry-picking when 'git cherry-pick' failed,
> so a failure made all further CLs in that bot fail.
>
> This CL fixes it and prevents further regressions.
>
> Original change's description:
> > gclient_scm: Use cherry-picking instead of rebasing.
> >
> > Currently gclient might include extra commits when applying patches.
> > For example, in this case we checkout |patch| and rebase it on top of |base|,
> > thus including an |extra commit| that we shouldn't.
> >
> > o master
> > |
> > . o patch
> > |/
> > o extra commit
> > |
> > o base (what gclient synced src at)
> >
> > This change uses the merge-base between |patch| and |master| to cherry-pick only
> > the changes belonging to the patch.
> >
> > Bug: 850812
> > Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
> > Reviewed-on: https://chromium-review.googlesource.com/1137052
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> > Reviewed-by: Aaron Gable <agable@chromium.org>
>
> Bug: 850812
> Change-Id: I83f38d0a258df3f5cd89e277f0d648badff29a22
> Reviewed-on: https://chromium-review.googlesource.com/1139554
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Bug: 850812
Change-Id: Ic65bda67c792bd7af5ec013a62d9615d1498eb3a
Reviewed-on: https://chromium-review.googlesource.com/1142805
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This reverts commit f9afc77f28.
Reason for revert: https://bugs.chromium.org/p/chromium/issues/detail?id=864301#c10
Original change's description:
> Reland "gclient_scm: Use cherry-picking instead of rebasing."
>
> The failures were caused by:
> 1 - When one change (call it #2) has been uploaded on top of another (#1),
> and (#1) has already landed, git cherry-pick complains that the range
> '<merge-base>..<change #2>' contains empty commits, since the contents
> of (#1) are already present in the tree.
> 2 - We did not abort the cherry-picking when 'git cherry-pick' failed,
> so a failure made all further CLs in that bot fail.
>
> This CL fixes it and prevents further regressions.
>
> Original change's description:
> > gclient_scm: Use cherry-picking instead of rebasing.
> >
> > Currently gclient might include extra commits when applying patches.
> > For example, in this case we checkout |patch| and rebase it on top of |base|,
> > thus including an |extra commit| that we shouldn't.
> >
> > o master
> > |
> > . o patch
> > |/
> > o extra commit
> > |
> > o base (what gclient synced src at)
> >
> > This change uses the merge-base between |patch| and |master| to cherry-pick only
> > the changes belonging to the patch.
> >
> > Bug: 850812
> > Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
> > Reviewed-on: https://chromium-review.googlesource.com/1137052
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> > Reviewed-by: Aaron Gable <agable@chromium.org>
>
> Bug: 850812
> Change-Id: I83f38d0a258df3f5cd89e277f0d648badff29a22
> Reviewed-on: https://chromium-review.googlesource.com/1139554
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
TBR=agable@chromium.org,ehmaldonado@chromium.org,jbudorick@chromium.org
Change-Id: I2138757899abd6eb9bece2dac3a25676effafdf3
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 850812
Reviewed-on: https://chromium-review.googlesource.com/1142624
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
The failures were caused by:
1 - When one change (call it #2) has been uploaded on top of another (#1),
and (#1) has already landed, git cherry-pick complains that the range
'<merge-base>..<change #2>' contains empty commits, since the contents
of (#1) are already present in the tree.
2 - We did not abort the cherry-picking when 'git cherry-pick' failed,
so a failure made all further CLs in that bot fail.
This CL fixes it and prevents further regressions.
Original change's description:
> gclient_scm: Use cherry-picking instead of rebasing.
>
> Currently gclient might include extra commits when applying patches.
> For example, in this case we checkout |patch| and rebase it on top of |base|,
> thus including an |extra commit| that we shouldn't.
>
> o master
> |
> . o patch
> |/
> o extra commit
> |
> o base (what gclient synced src at)
>
> This change uses the merge-base between |patch| and |master| to cherry-pick only
> the changes belonging to the patch.
>
> Bug: 850812
> Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
> Reviewed-on: https://chromium-review.googlesource.com/1137052
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>
Bug: 850812
Change-Id: I83f38d0a258df3f5cd89e277f0d648badff29a22
Reviewed-on: https://chromium-review.googlesource.com/1139554
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This reverts commit fb78b368de.
Reason for revert: https://bugs.chromium.org/p/chromium/issues/detail?id=864301
Original change's description:
> gclient_scm: Use cherry-picking instead of rebasing.
>
> Currently gclient might include extra commits when applying patches.
> For example, in this case we checkout |patch| and rebase it on top of |base|,
> thus including an |extra commit| that we shouldn't.
>
> o master
> |
> . o patch
> |/
> o extra commit
> |
> o base (what gclient synced src at)
>
> This change uses the merge-base between |patch| and |master| to cherry-pick only
> the changes belonging to the patch.
>
> Bug: 850812
> Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
> Reviewed-on: https://chromium-review.googlesource.com/1137052
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>
TBR=agable@chromium.org,ehmaldonado@chromium.org
Change-Id: I57299e60e58eac5656dc88937c622d0d14c4ba37
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 850812
Reviewed-on: https://chromium-review.googlesource.com/1139553
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
Currently gclient might include extra commits when applying patches.
For example, in this case we checkout |patch| and rebase it on top of |base|,
thus including an |extra commit| that we shouldn't.
o master
|
. o patch
|/
o extra commit
|
o base (what gclient synced src at)
This change uses the merge-base between |patch| and |master| to cherry-pick only
the changes belonging to the patch.
Bug: 850812
Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
Reviewed-on: https://chromium-review.googlesource.com/1137052
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
The update_scripts step doesn't set '--reset'
Additionally, if there was no fetch config, don't fail trying to unset it.
Tbr: agable@chromium.org
No-Try: True
Bug: 862547
Change-Id: I90886ca7d1dd20ae59b378a5998de47dc67c60f9
Reviewed-on: https://chromium-review.googlesource.com/1137693
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
When a git mirror is configured, it fetches only refs/heads/*,
and possibly branch-heads and tags, but not refs/changes.
When gclient attempts to sync refs/changes/* from a mirror,
it fails, since the mirror has no such objects.
See for example [1], where the DEPS entry for builtools was
modified to sync to a CL, but all the bots failed to sync to it.
This change modifies gclient to always fetch refs/changes/*
directly from the repo, even when a mirror is configured.
[1] https://chromium-review.googlesource.com/c/chromium/src/+/1119098/6
Bug: 858894
Change-Id: I71bb313e4325a81b2985db1d00c70a8844dc7c22
Reviewed-on: https://chromium-review.googlesource.com/1119525
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
This CL makes a couple changes:
* The goal here is to be able to specify git cache entirely from the
environment variable $GIT_CACHE_PATH and not require special treatment from
bot_update. Eventually this will be specified at the swarming task level
instead of in the recipe (i.e. "cached git" will eventually be an
implementation detail of git on the bots and completely transparent to
all other software).
* Removal of the general --cache-dir option from gclient. This option was
error-prone; it doesn't actually make sense to configure this on
a per-invocation basis. The sole exception was `gclient config`, which
now allows this option to be set directly.
* Consolidation of GitWrapper.cache_dir and GitWrapper._GetMirror; previously
these two things could disagree, leading to weird intermediate states. Now
they're the same value.
R=agable@chromium.org, ehmaldonado@chromium.org, tandrii@chromium.org
Bug: 823434
Change-Id: I14adc7619b5fc10768ce32be2651c6215ba94aff
Reviewed-on: https://chromium-review.googlesource.com/1105473
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
John Budorick
Edward Lesmes
Edward Lesmes
Anthony Polito
danakj
Robert Iannucci
Dirk Pranke
Erik Chen
Mike Frysinger
Quinten Yearsley
Raul Tambre
Aaron Gable
Henrique Ferreiro
Michael Spang