Commit Graph

385 Commits (e37b1ec5e44ed4b4698331a1d5b42dbe1dc0d640)

Author SHA1 Message Date
Quinten Yearsley 3841eaa164 Remove unused imports in depot_tools/*.py
I know that sometimes imports can have side-effects,
so unused imports shouldn't always be removed, but these
ones look like they could be.

Change-Id: Iea9f82afa99b0ea35f29a28f20ce0493b579cfee
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1819860
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
6 years ago
Edward Lemur 26a8b9f356 depot_tools: Make gclient Python3 compatible.
Bug: 984182
Change-Id: Idc38678acbfca9cc6b89d77db28fdc24784f2b7d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1753742
Commit-Queue: Edward Lemur <ehmaldonado@google.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
6 years ago
Edward Lemur 979fa780ce depot_tools: Make gclient_scm Python 3 compatible.
Bug: 984182
Change-Id: Ib9c5d2762546f29eaca5eae89b0428431067da4a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1753029
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
6 years ago
Edward Lemur 24146be833 depot_tools: Simplify CheckCallAndFilter[AndHeader]
- Merge CheckCallAndFilter[AndHeader]
- print_stdout will cause command output to be redirected to sys.stdout.
- Made compatible with Python 3.

Bug: 984182
Change-Id: Ida30e295b872c8c1a1474a376a90aecea924f307
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1727404
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
6 years ago
Edward Lemur 4c5c8ab626 Reland "Reland "gclient: Require a target ref when applying patches.""
This is a reland of 822dbc51a2

Skia has fixed the test.

Original change's description:
> Reland "gclient: Require a target ref when applying patches."
>
> This is a reland of 1d6478a5ff
>
> Original change's description:
> > gclient: Require a target ref when applying patches.
> >
> > Bug: 956807
> > Change-Id: Icfffe965f9f4651f22e8ba32c60133a5620bb350
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1616804
> > Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
>
> Bug: 956807
> Change-Id: I3de8475a091ce6a2a14ff7dcfb92507a205ef78c
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1623594
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

TBR=tandrii@chromium.org

Bug: 956807
Change-Id: Ie273dafb921206f6346678c1861320129ad5b6fd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1649695
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Edward Lesmes 42b03c3453 Revert "Reland "gclient: Require a target ref when applying patches.""
This reverts commit 822dbc51a2.

Reason for revert:
Skia autoroller is complaining
https://skia-review.googlesource.com/c/buildbot/+/219338

Original change's description:
> Reland "gclient: Require a target ref when applying patches."
> 
> This is a reland of 1d6478a5ff
> 
> Original change's description:
> > gclient: Require a target ref when applying patches.
> > 
> > Bug: 956807
> > Change-Id: Icfffe965f9f4651f22e8ba32c60133a5620bb350
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1616804
> > Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> 
> Bug: 956807
> Change-Id: I3de8475a091ce6a2a14ff7dcfb92507a205ef78c
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1623594
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

TBR=borenet@chromium.org,tandrii@chromium.org,ehmaldonado@chromium.org

Change-Id: I14a70206a34f7024abc21039806f23dc7cf56bb6
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 956807
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1648321
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Edward Lemur 822dbc51a2 Reland "gclient: Require a target ref when applying patches."
This is a reland of 1d6478a5ff

Original change's description:
> gclient: Require a target ref when applying patches.
> 
> Bug: 956807
> Change-Id: Icfffe965f9f4651f22e8ba32c60133a5620bb350
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1616804
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

Bug: 956807
Change-Id: I3de8475a091ce6a2a14ff7dcfb92507a205ef78c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1623594
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Edward Lemur 3acbc749e4 gclient: Make it possible to set target_rev to a commit hash.
Bug: 956807
Change-Id: I333479715257a582e26a823ed5c0e2382851ff35
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1625775
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
6 years ago
Edward Lesmes 9a01eb27ff Revert "gclient: Require a target ref when applying patches."
This reverts commit 1d6478a5ff.

Reason for revert: 
Skia doesn't pass target refs to patches yet.

Original change's description:
> gclient: Require a target ref when applying patches.
> 
> Bug: 956807
> Change-Id: Icfffe965f9f4651f22e8ba32c60133a5620bb350
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1616804
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

TBR=tandrii@chromium.org,ehmaldonado@chromium.org

Change-Id: I96c7d4ab7df151f50e82f783ef1a8769890aef9a
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 956807
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1623593
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Edward Lemur 1d6478a5ff gclient: Require a target ref when applying patches.
Bug: 956807
Change-Id: Icfffe965f9f4651f22e8ba32c60133a5620bb350
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1616804
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Raul Tambre cd862e37ea gclient_scm: Decode Git output centrally for better Python 3 support
This avoids the need to do the decoding everywhere separately.
Also a small fix for filter() no longer returning a list on Py3.

The scripts still work with Python 2.
There are no intended behaviour changes.

Bug: 942522
Change-Id: Id0f5153011b2ef1b64394359087864cd9434e45e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1595685
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
Auto-Submit: Raul Tambre <raul@tambre.ee>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
6 years ago
Edward Lemur 8c66565649 gclient: Fix applying patches to branch heads.
Bug: 956807
Change-Id: I2eed6db1f338806812fca778986d51b0a007eddd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1592577
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
6 years ago
Edward Lemur a0ffbe4505 gclient: Convert target ref to patch to a remote ref.
Bug: 956807
Change-Id: I2e1d0b61d99796434115012709b6e79e507e2986
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1590495
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Aaron Gable ac9b0f3786 Restore py2/py3 compatibility with basestring
A previous CL replaced "basestring" with "str", because basestring
does not exist in Python 3. However, this broke Python 2's ability
to interoperate with unicode strings. This CL introduces a workaround
(defining basestring to be equivalent to string, if it doesn't exist
already), and restores the references to basestring. This workaround
can be fixed when we're 100% on Python 3.

It also undoes some unnecessary and harder-to-read formatting changes.

Bug: 942522
Change-Id: I4a31ee46dc048134c2e4832b6c44ea00ce341899
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1572441
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
6 years ago
Henrique Ferreiro e72279d25f Keep warning about unversioned trees until removed
Restore previous behavior and keep emitting a warning when an existing
gclient entry is not longer part of the client. Additionally, suggest
the usage of `gclient sync -D` to automatically remove them.

Change-Id: I62d4662ae4d0886d340230019419b68debffc5ba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1483031
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Henrique Ferreiro <hferreiro@igalia.com>
6 years ago
Edward Lemur 9a5e3bd817 gclient_scm: Simplify fetching refs.
Bug: 874501, 942229
Change-Id: Ie9896e8a289e32a1b468ed5fa51c95a81970bdf8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1544802
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
6 years ago
Edward Lemur bdbe07f29d gclient: Support syncing deps to refs/{branch-heads,tags}:revision.
Arbitrary refs (i.e. syncing to refs/arbitrary/stuff:revision) is not
guaranteed to work.

Bug: 874501, 942229
Change-Id: If08f0d9ae257e3d9f486d3ea6b310c8d94d3b918
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1542588
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Karen Qian <karenqian@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Raul Tambre b946b2389c Improve gclient Python 3 compatibility
This enables gclient sync and gclient runhooks to run, barring hook script failures.
git cl upload also now works.

The scripts still work with Python 2.
There are no intended behaviour changes.

Bug: 942522
Change-Id: I2ac587b5f803ba7f5bb5e412337ce049f4b1a741
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1524583
Commit-Queue: Raul Tambre <raul@tambre.ee>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
6 years ago
Raul Tambre c2f74c1698 Make fetch compatible with Python 3
The scripts still work with Python 2.
There are no intended behaviour changes.

Bug: 939847
Change-Id: Icada60c5b2cf351d62aead26b7364fcef2c2a3e5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1524486
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Raul Tambre <raul@tambre.ee>
6 years ago
Michael Spang 73fac91fc5 Avoid unnecessary invocations of "git fetch" (reland)
If we already have the correct commit, it is not necessary to fetch.  Any
fetches can takes minutes due to gerrit problems, and often fail
completely, so it is helpful to avoid them completely whenever we can.

Reland now that skia autoroller does its own fetch.

Bug: 938627

Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1504102
Commit-Queue: Michael Spang <spang@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>

Change-Id: I5a6877125c682ff0d98bd6d2ffe495245f74c513
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1512024
Auto-Submit: Michael Spang <spang@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Edward Lesmes 0cec542e04 Revert "Avoid unnecessary invocations of "git fetch""
This reverts commit 61a1b5f310.

Reason for revert:
It's blocking skia autoroller
https://skia-review.googlesource.com/c/buildbot/+/198700

Original change's description:
> Avoid unnecessary invocations of "git fetch"
> 
> If we already have the correct commit, it is not necessary to fetch.  Any
> fetches can takes minutes due to gerrit problems, and often fail
> completely, so it is helpful to avoid them completely whenever we can.
> 
> Bug: 938627
> 
> Change-Id: I54494cad9b9aa0090f51674b6b9f56bb3e842f12
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1504102
> Commit-Queue: Michael Spang <spang@chromium.org>
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>

TBR=spang@chromium.org,agable@chromium.org,ehmaldonado@chromium.org

Change-Id: If8bd26d74bf84ff0ca69378dbfe6b9369b9988d3
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 938627
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1508980
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Edward Lemur db5c5ade84 gclient: Fetch refs/{branch-heads,tags}/* if needed.
If we're asked to sync a repo to branch-heads or tags,
make sure we fetch refs/branch-heads/* and refs/tags/* even if
--with-branch-heads or --with-tags was not passed.

Bug: 938160
Change-Id: I04de8e7762c83322692d9551d057ec803fd2ac9c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1506420
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
Michael Spang 61a1b5f310 Avoid unnecessary invocations of "git fetch"
If we already have the correct commit, it is not necessary to fetch.  Any
fetches can takes minutes due to gerrit problems, and often fail
completely, so it is helpful to avoid them completely whenever we can.

Bug: 938627

Change-Id: I54494cad9b9aa0090f51674b6b9f56bb3e842f12
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1504102
Commit-Queue: Michael Spang <spang@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
6 years ago
John Budorick 21a51b3921 gclient: set the push url to the actual repository instead of the mirror.
Bug: 884434
Change-Id: Id156709e14d1c3506a1236bc1e59f8468381a402
Reviewed-on: https://chromium-review.googlesource.com/1232257
Commit-Queue: John Budorick <jbudorick@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur 6a4e31b488 gclient_scm: Add option to specify the branch the patch is based on.
Also add shortcuts, so we don't have to look through all the branches
if the patch is based on a common branch (i.e. master, infra/config, lkgr)


Bug: 870279
Change-Id: I625a8481dccac9a475b096b926e6fab7efe676b0
Reviewed-on: https://chromium-review.googlesource.com/1161094
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Edward Lemur ca7d881540 Reland "Reland "gclient_scm: Use cherry-picking instead of rebasing.""
Abort any cherry-picks before applying the patch, so that if the bots are in a
bad state, we don't fail.

Original change's description:
> Reland "gclient_scm: Use cherry-picking instead of rebasing."
>
> The failures were caused by:
>  1 - When one change (call it #2) has been uploaded on top of another (#1),
>      and (#1) has already landed, git cherry-pick complains that the range
>      '<merge-base>..<change #2>' contains empty commits, since the contents
>      of (#1) are already present in the tree.
>  2 - We did not abort the cherry-picking when 'git cherry-pick' failed,
>      so a failure made all further CLs in that bot fail.
>
> This CL fixes it and prevents further regressions.
>
> Original change's description:
> > gclient_scm: Use cherry-picking instead of rebasing.
> >
> > Currently gclient might include extra commits when applying patches.
> > For example, in this case we checkout |patch| and rebase it on top of |base|,
> > thus including an |extra commit| that we shouldn't.
> >
> > o master
> > |
> > . o patch
> > |/
> > o extra commit
> > |
> > o base (what gclient synced src at)
> >
> > This change uses the merge-base between |patch| and |master| to cherry-pick only
> > the changes belonging to the patch.
> >
> > Bug: 850812
> > Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
> > Reviewed-on: https://chromium-review.googlesource.com/1137052
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> > Reviewed-by: Aaron Gable <agable@chromium.org>
>
> Bug: 850812
> Change-Id: I83f38d0a258df3f5cd89e277f0d648badff29a22
> Reviewed-on: https://chromium-review.googlesource.com/1139554
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

Bug: 850812
Change-Id: Ic65bda67c792bd7af5ec013a62d9615d1498eb3a
Reviewed-on: https://chromium-review.googlesource.com/1142805
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
John Budorick 2c984a03ca Revert "Reland "gclient_scm: Use cherry-picking instead of rebasing.""
This reverts commit f9afc77f28.

Reason for revert: https://bugs.chromium.org/p/chromium/issues/detail?id=864301#c10

Original change's description:
> Reland "gclient_scm: Use cherry-picking instead of rebasing."
> 
> The failures were caused by:
>  1 - When one change (call it #2) has been uploaded on top of another (#1),
>      and (#1) has already landed, git cherry-pick complains that the range
>      '<merge-base>..<change #2>' contains empty commits, since the contents
>      of (#1) are already present in the tree.
>  2 - We did not abort the cherry-picking when 'git cherry-pick' failed,
>      so a failure made all further CLs in that bot fail.
> 
> This CL fixes it and prevents further regressions.
> 
> Original change's description:
> > gclient_scm: Use cherry-picking instead of rebasing.
> >
> > Currently gclient might include extra commits when applying patches.
> > For example, in this case we checkout |patch| and rebase it on top of |base|,
> > thus including an |extra commit| that we shouldn't.
> >
> > o master
> > |
> > . o patch
> > |/
> > o extra commit
> > |
> > o base (what gclient synced src at)
> >
> > This change uses the merge-base between |patch| and |master| to cherry-pick only
> > the changes belonging to the patch.
> >
> > Bug: 850812
> > Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
> > Reviewed-on: https://chromium-review.googlesource.com/1137052
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> > Reviewed-by: Aaron Gable <agable@chromium.org>
> 
> Bug: 850812
> Change-Id: I83f38d0a258df3f5cd89e277f0d648badff29a22
> Reviewed-on: https://chromium-review.googlesource.com/1139554
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

TBR=agable@chromium.org,ehmaldonado@chromium.org,jbudorick@chromium.org

Change-Id: I2138757899abd6eb9bece2dac3a25676effafdf3
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 850812
Reviewed-on: https://chromium-review.googlesource.com/1142624
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
7 years ago
Edward Lemur f9afc77f28 Reland "gclient_scm: Use cherry-picking instead of rebasing."
The failures were caused by:
 1 - When one change (call it #2) has been uploaded on top of another (#1),
     and (#1) has already landed, git cherry-pick complains that the range
     '<merge-base>..<change #2>' contains empty commits, since the contents
     of (#1) are already present in the tree.
 2 - We did not abort the cherry-picking when 'git cherry-pick' failed,
     so a failure made all further CLs in that bot fail.

This CL fixes it and prevents further regressions.

Original change's description:
> gclient_scm: Use cherry-picking instead of rebasing.
>
> Currently gclient might include extra commits when applying patches.
> For example, in this case we checkout |patch| and rebase it on top of |base|,
> thus including an |extra commit| that we shouldn't.
>
> o master
> |
> . o patch
> |/
> o extra commit
> |
> o base (what gclient synced src at)
>
> This change uses the merge-base between |patch| and |master| to cherry-pick only
> the changes belonging to the patch.
>
> Bug: 850812
> Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
> Reviewed-on: https://chromium-review.googlesource.com/1137052
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>

Bug: 850812
Change-Id: I83f38d0a258df3f5cd89e277f0d648badff29a22
Reviewed-on: https://chromium-review.googlesource.com/1139554
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
John Budorick 302bb847d3 gclient: run cipd ensure in paranoid mode.
Bug: 860251
Change-Id: I0acaafcfe1d214d102b6a17b29b76aadcc1542c2
Reviewed-on: https://chromium-review.googlesource.com/1132345
Commit-Queue: John Budorick <jbudorick@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
7 years ago
John Budorick 40bacee96a Revert "gclient_scm: Use cherry-picking instead of rebasing."
This reverts commit fb78b368de.

Reason for revert: https://bugs.chromium.org/p/chromium/issues/detail?id=864301

Original change's description:
> gclient_scm: Use cherry-picking instead of rebasing.
> 
> Currently gclient might include extra commits when applying patches.
> For example, in this case we checkout |patch| and rebase it on top of |base|,
> thus including an |extra commit| that we shouldn't.
> 
> o master
> |
> . o patch
> |/
> o extra commit
> |
> o base (what gclient synced src at)
> 
> This change uses the merge-base between |patch| and |master| to cherry-pick only
> the changes belonging to the patch.
> 
> Bug: 850812
> Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
> Reviewed-on: https://chromium-review.googlesource.com/1137052
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>

TBR=agable@chromium.org,ehmaldonado@chromium.org

Change-Id: I57299e60e58eac5656dc88937c622d0d14c4ba37
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 850812
Reviewed-on: https://chromium-review.googlesource.com/1139553
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
7 years ago
Edward Lemur fb78b368de gclient_scm: Use cherry-picking instead of rebasing.
Currently gclient might include extra commits when applying patches.
For example, in this case we checkout |patch| and rebase it on top of |base|,
thus including an |extra commit| that we shouldn't.

o master
|
. o patch
|/
o extra commit
|
o base (what gclient synced src at)

This change uses the merge-base between |patch| and |master| to cherry-pick only
the changes belonging to the patch.

Bug: 850812
Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
Reviewed-on: https://chromium-review.googlesource.com/1137052
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
7 years ago
Edward Lemur 2f38df65f9 gclient: Reset the fetch config also when --force is passed.
The update_scripts step doesn't set '--reset'
Additionally, if there was no fetch config, don't fail trying to unset it.

Tbr: agable@chromium.org
No-Try: True
Bug: 862547
Change-Id: I90886ca7d1dd20ae59b378a5998de47dc67c60f9
Reviewed-on: https://chromium-review.googlesource.com/1137693
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur 579c98612b Reset the fetch config in the bots before each run.
If a ref is incorrectly set, it will affect later runs.
For example, [1] caused a bad refspec to be added to the fetch config,
(+refs/branch-heads/refs/branch-heads/6.7:refs/branch-heads/refs/branch-heads/6.7)
which affects all subsequent runs in the same bot: [2], [3].

This makes bot_update clear the fetch config before each run, so it is more
robust against these kind of failures.

[1] https://logs.chromium.org/v/?s=v8%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8942033142534034848%2F%2B%2Fsteps%2Fbot_update%2F0%2Fstdout
[2] https://logs.chromium.org/v/?s=v8%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8941307871443322944%2F%2B%2Fsteps%2Fbot_update%2F0%2Fstdout
[3] https://logs.chromium.org/v/?s=v8%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8941333453153279680%2F%2B%2Fsteps%2Fbot_update%2F0%2Fstdout

Bug: 862547
Change-Id: I2f849c604656e81ebd7377465d287226b8bdea1a
Reviewed-on: https://chromium-review.googlesource.com/1135645
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
7 years ago
John Budorick 882c91ed97 Revert "gclient: Add support for the branch:revision format."
This reverts commit d328b47eb1.

Reason for revert: suspect this caused https://bugs.chromium.org/p/chromium/issues/detail?id=863211

Original change's description:
> gclient: Add support for the branch:revision format.
> 
> Bug: 850812, 853032
> Change-Id: I597acbde2b3c229813b7eba8fcba52d5877130b2
> Reviewed-on: https://chromium-review.googlesource.com/1119235
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>

TBR=agable@chromium.org,tandrii@chromium.org,ehmaldonado@chromium.org

Change-Id: I1e4c00b83a2840cc5a87621469da866dad58a20c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 850812, 853032
Reviewed-on: https://chromium-review.googlesource.com/1135893
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
7 years ago
Edward Lemur d328b47eb1 gclient: Add support for the branch:revision format.
Bug: 850812, 853032
Change-Id: I597acbde2b3c229813b7eba8fcba52d5877130b2
Reviewed-on: https://chromium-review.googlesource.com/1119235
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
7 years ago
Edward Lemur d64781e17a gclient_scm: Fetch refs/changes/* when syncing.
When a git mirror is configured, it fetches only refs/heads/*,
and possibly branch-heads and tags, but not refs/changes.

When gclient attempts to sync refs/changes/* from a mirror,
it fails, since the mirror has no such objects.

See for example [1], where the DEPS entry for builtools was
modified to sync to a CL, but all the bots failed to sync to it.

This change modifies gclient to always fetch refs/changes/*
directly from the repo, even when a mirror is configured.

[1] https://chromium-review.googlesource.com/c/chromium/src/+/1119098/6

Bug: 858894

Change-Id: I71bb313e4325a81b2985db1d00c70a8844dc7c22
Reviewed-on: https://chromium-review.googlesource.com/1119525
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
7 years ago
Robert Iannucci a19649b86b Clean up cache_dir handling in gclient/git_cache/bot_update.
This CL makes a couple changes:
  * The goal here is to be able to specify git cache entirely from the
    environment variable $GIT_CACHE_PATH and not require special treatment from
    bot_update. Eventually this will be specified at the swarming task level
    instead of in the recipe (i.e. "cached git" will eventually be an
    implementation detail of git on the bots and completely transparent to
    all other software).
  * Removal of the general --cache-dir option from gclient. This option was
    error-prone; it doesn't actually make sense to configure this on
    a per-invocation basis. The sole exception was `gclient config`, which
    now allows this option to be set directly.
  * Consolidation of GitWrapper.cache_dir and GitWrapper._GetMirror; previously
    these two things could disagree, leading to weird intermediate states. Now
    they're the same value.

R=agable@chromium.org, ehmaldonado@chromium.org, tandrii@chromium.org

Bug: 823434
Change-Id: I14adc7619b5fc10768ce32be2651c6215ba94aff
Reviewed-on: https://chromium-review.googlesource.com/1105473
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
7 years ago
Andrii Shyshkalov 690d8d437b Revert "gclient_scm: Use cherry-picking instead of rebasing."
This reverts commit c912114140.

Reason for revert: broke patch application on infra/config https://crbug.com/853032

Original change's description:
> gclient_scm: Use cherry-picking instead of rebasing.
> 
> We have a problem when in this situation, we checkout |patch| and rebase it on
> top of |base|, thus including an |extra commit| that we shouldn't.
> 
> o master
> |
> . o patch
> |/
> o extra commit
> |
> o base (what gclient synced src at)
> 
> This does merge-base between |patch| and |master|, and cherry-picks only the
> changes belonging to the patch.
> 
> Bug: 850812
> Change-Id: Id09ae1682e53b69ed49b2fb649310de6a6a8a29e
> Reviewed-on: https://chromium-review.googlesource.com/1098228
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>

TBR=agable@chromium.org,ehmaldonado@chromium.org

Change-Id: Ib3feeee2f44f5441713383f1dbf08db16fae4717
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 850812, 853032
Reviewed-on: https://chromium-review.googlesource.com/1101977
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Edward Lemur c912114140 gclient_scm: Use cherry-picking instead of rebasing.
We have a problem when in this situation, we checkout |patch| and rebase it on
top of |base|, thus including an |extra commit| that we shouldn't.

o master
|
. o patch
|/
o extra commit
|
o base (what gclient synced src at)

This does merge-base between |patch| and |master|, and cherry-picks only the
changes belonging to the patch.

Bug: 850812
Change-Id: Id09ae1682e53b69ed49b2fb649310de6a6a8a29e
Reviewed-on: https://chromium-review.googlesource.com/1098228
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
7 years ago
Edward Lemur fbb06aaab1 gclient: Use only recursedeps to decide whether to process a dependency.
Bug: 839925
Change-Id: If86cfbdbea795b6da53dec1ea6e44c5af8a3b846
Reviewed-on: https://chromium-review.googlesource.com/1086270
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur e7273d2501 Reland "gclient: Get rid of parsed_url."
This is a reland of e877b1776a

Original change's description:
> gclient: Get rid of parsed_url.
>
> There is no reason I can see to set parsed_url so late.
> Also, the tests are misleading, since relative URLs don't behave the way
> the tests led you to believe.
>
> Bug: 839925
> Change-Id: I08d92b7b7847bdc406f003d4a4139d968cc662b1
> Reviewed-on: https://chromium-review.googlesource.com/1047797
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>

TBR=agable@chromium.org

Bug: 839925
Change-Id: I9200ec5fbe7289022e9754f0c78676dc931fcaeb
Reviewed-on: https://chromium-review.googlesource.com/1054567
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
7 years ago
John Budorick 2191308678 Revert "gclient: Get rid of parsed_url."
Suspected of breaking the world.

This reverts commit e877b1776a.

TBR=ehmaldonado@chromium.org,tandrii@chromium.org
Bug: 841936
Change-Id: Iad2b55a2235d8d0b1a3d7681cbd577f795cb89dd
Reviewed-on: https://chromium-review.googlesource.com/1054440
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
7 years ago
Edward Lemur e877b1776a gclient: Get rid of parsed_url.
There is no reason I can see to set parsed_url so late.
Also, the tests are misleading, since relative URLs don't behave the way
the tests led you to believe.

Bug: 839925
Change-Id: I08d92b7b7847bdc406f003d4a4139d968cc662b1
Reviewed-on: https://chromium-review.googlesource.com/1047797
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
7 years ago
Edward Lesmes 97ce05851c gclient_scm: Try to rebase after printing what was wrong.
So we know why did git fail on the Chromium Windows trybots.

R=agable@chromium.org

Bug: 643346
Change-Id: Iac0abd9202fb56fbed6f3edf382b755e96fed3bf
Reviewed-on: https://chromium-review.googlesource.com/1031128
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lesmes 8271d22542 gclient_scm: Don't try to update file_list if it's None.
file_list can be none if gclient runs with --nohooks.
Fix apply_patch_ref so that it doesn't try to update file_list when it is
None.

TBR=agable@chromium.org

Bug: 643346
Change-Id: If00547da004415edfe68196a44cbda753b4db017
Reviewed-on: https://chromium-review.googlesource.com/989279
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lesmes c621b21ad4 gclient sync: Add support to apply gerrit refs.
Mimics bot_update's functionality to apply gerrit refs in gclient
via --gerrit-ref flags.
When the patch fails to apply, gclient sync will return exit code 2.

The idea is to move this logic from bot_update to gclient sync to
deal when patches for projects like ANGLE are tried on Chromium bots.
This way the patch is applied before recursively parsing and syncing
ANGLE’s DEPS.chromium file, which doesn't currently happen.

Bug: 643346

Change-Id: I7e2018b3c393a5ac9852b8c3611f906977eeeb18
Reviewed-on: https://chromium-review.googlesource.com/961605
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
John Budorick d3ba72bbc6 [gclient] Use a single CIPD root for an entire gclient checkout.
gclient previously used a CIPD root per DEPS file. This didn't work
in cases in which a DEPS file wanted to specify a CIPD package outside
of its directory hierarchy, though, as is the case with buildspecs.

Bug: 755920
Change-Id: I0d6c3db567f17f9171c0feaaf9ed6bc64db22757
Reviewed-on: https://chromium-review.googlesource.com/955933
Commit-Queue: John Budorick <jbudorick@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
7 years ago
Aaron Gable 1853f66efd gclient: diff against hash in DEPS
When someone runs "gclient diff", they expect to be diffing their
local state against the state of a clean gclient checkout.

Up until now, "gclient diff" has been doing... not that. It's been
assuming that the sha1 in DEPS is approximately the head of the
remote's master branch, finding the merge-base between the local
content and that remote master, and diffing against that merge-base.
This makes basically no sense in the context of gclient. With
gclient, we already know exactly what should be checked out: it's
whatever is in DEPS.

This CL updates the behavior of "gclient diff" to do the right thing.

Bug: 731701
Change-Id: I23a43cbc0d6bc7772495097ff615d4d070f5ad19
Reviewed-on: https://chromium-review.googlesource.com/914717
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
7 years ago
John Budorick 3929e9ee94 Add a name attribute to CipdWrapper.
TBR=agable@chromium.org

Bug: 789809
Change-Id: I4ede72a3584b23d01063e5cfa20401c921762b50
Reviewed-on: https://chromium-review.googlesource.com/900664
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
7 years ago
Edward Lemur 231f5eab45 Print git output when cloning a repo.
This will allow us to show the progress when cloning a new repo when running
commands like gclient sync.

R=agable@chromium.org

Bug: 722686
Change-Id: I268cba343ea4c3c024292c9341d5009aa112b184
Reviewed-on: https://chromium-review.googlesource.com/890524
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
7 years ago