When a git mirror is configured, it fetches only refs/heads/*,
and possibly branch-heads and tags, but not refs/changes.
When gclient attempts to sync refs/changes/* from a mirror,
it fails, since the mirror has no such objects.
See for example [1], where the DEPS entry for builtools was
modified to sync to a CL, but all the bots failed to sync to it.
This change modifies gclient to always fetch refs/changes/*
directly from the repo, even when a mirror is configured.
[1] https://chromium-review.googlesource.com/c/chromium/src/+/1119098/6
Bug: 858894
Change-Id: I71bb313e4325a81b2985db1d00c70a8844dc7c22
Reviewed-on: https://chromium-review.googlesource.com/1119525
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
This reverts commit c912114140.
Reason for revert: broke patch application on infra/config https://crbug.com/853032
Original change's description:
> gclient_scm: Use cherry-picking instead of rebasing.
>
> We have a problem when in this situation, we checkout |patch| and rebase it on
> top of |base|, thus including an |extra commit| that we shouldn't.
>
> o master
> |
> . o patch
> |/
> o extra commit
> |
> o base (what gclient synced src at)
>
> This does merge-base between |patch| and |master|, and cherry-picks only the
> changes belonging to the patch.
>
> Bug: 850812
> Change-Id: Id09ae1682e53b69ed49b2fb649310de6a6a8a29e
> Reviewed-on: https://chromium-review.googlesource.com/1098228
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>
TBR=agable@chromium.org,ehmaldonado@chromium.org
Change-Id: Ib3feeee2f44f5441713383f1dbf08db16fae4717
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 850812, 853032
Reviewed-on: https://chromium-review.googlesource.com/1101977
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
We have a problem when in this situation, we checkout |patch| and rebase it on
top of |base|, thus including an |extra commit| that we shouldn't.
o master
|
. o patch
|/
o extra commit
|
o base (what gclient synced src at)
This does merge-base between |patch| and |master|, and cherry-picks only the
changes belonging to the patch.
Bug: 850812
Change-Id: Id09ae1682e53b69ed49b2fb649310de6a6a8a29e
Reviewed-on: https://chromium-review.googlesource.com/1098228
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Mimics bot_update's functionality to apply gerrit refs in gclient
via --gerrit-ref flags.
When the patch fails to apply, gclient sync will return exit code 2.
The idea is to move this logic from bot_update to gclient sync to
deal when patches for projects like ANGLE are tried on Chromium bots.
This way the patch is applied before recursively parsing and syncing
ANGLE’s DEPS.chromium file, which doesn't currently happen.
Bug: 643346
Change-Id: I7e2018b3c393a5ac9852b8c3611f906977eeeb18
Reviewed-on: https://chromium-review.googlesource.com/961605
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
gclient previously used a CIPD root per DEPS file. This didn't work
in cases in which a DEPS file wanted to specify a CIPD package outside
of its directory hierarchy, though, as is the case with buildspecs.
Bug: 755920
Change-Id: I0d6c3db567f17f9171c0feaaf9ed6bc64db22757
Reviewed-on: https://chromium-review.googlesource.com/955933
Commit-Queue: John Budorick <jbudorick@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
This includes a minor refactor so that some gclient_scm methods
can all share the same core.quotePath specifier.
R=iannucci
Bug: 792302
Change-Id: Iaadf190f5c0666787cf7c2ccda88d6dba9aace9b
Reviewed-on: https://chromium-review.googlesource.com/823131
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
This is a reland of 9219d35688
The original was reverted due to a typo (core,quotePath instead
of core.quotePath). This version is fixed.
Original change's description:
> Use core.quotePath=false when git is listing files
>
> This prevents git from putting quotes around some file names
> (those that have astral-plane characters) and not around others.
>
> R=maruel
>
> Bug: 792302
>
> Change-Id: I3b6a6b36c4720116de811b40177b59aa25c263db
> Reviewed-on: https://chromium-review.googlesource.com/815454
> Commit-Queue: Aaron Gable <agable@chromium.org>
> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
Bug: 792302
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Change-Id: I28d2260948aaf63bd865888c2f60e4cdee9aea48
Reviewed-on: https://chromium-review.googlesource.com/822990
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This reverts commit 9219d35688.
Reason for revert: unfortunately this says "core,quotePath" and since it includes recipe changes, we need something that the roller can munch on :(
Original change's description:
> Use core.quotePath=false when git is listing files
>
> This prevents git from putting quotes around some file names
> (those that have astral-plane characters) and not around others.
>
> R=maruel
>
> Bug: 792302
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: build_limited_scripts_slave
> Change-Id: I3b6a6b36c4720116de811b40177b59aa25c263db
> Reviewed-on: https://chromium-review.googlesource.com/815454
> Commit-Queue: Aaron Gable <agable@chromium.org>
> Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
TBR=maruel@chromium.org,agable@chromium.org
Change-Id: I226388f19024403240a1443eb2b878b9293220e1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 792302
Reviewed-on: https://chromium-review.googlesource.com/821671
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
This prevents git from putting quotes around some file names
(those that have astral-plane characters) and not around others.
R=maruel
Bug: 792302
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Change-Id: I3b6a6b36c4720116de811b40177b59aa25c263db
Reviewed-on: https://chromium-review.googlesource.com/815454
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
The newline is probably here to protect against a hook that doesn't print
a trailing newline. I've never seen a hook that did that, and if we found one,
we could make the logic look like "print a trailing newline if it's not there"
-- or just fix the hook.
The newline has been around since depot_tools was created
(https://codereview.chromium.org/92087, gclient.py), back then this was in
a general "run stuff" function, not in hooks-specific code. Maybe it made
more sense back then.
This is part of a few changes to make `gclient runhooks` less noisy.
Bug: 772741
Change-Id: I285f76dc3f01c5acf5bbaa0be4db9f6edb9c0366
Reviewed-on: https://chromium-review.googlesource.com/706914
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Nico Weber <thakis@chromium.org>
We were checking for output which has been dropped from 2.11.0, so loosen the
test a bit.
BUG=670678
Change-Id: Ic610c76ceed4ab42a3b9f2bb8b952a3689658cbf
Reviewed-on: https://chromium-review.googlesource.com/416402
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
This affects a bunch of files, but only changes comments,
and shouldn't make any difference to behavior.
The purpose is to slightly improve readability of pylint
disable comments.
Change-Id: Ic6cd0f8de792b31d91c6125f6da2616450b30f11
Reviewed-on: https://chromium-review.googlesource.com/420412
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
This removes SVN support (most notably the SVNWrapper class, and the git-svn
logic in GitWrapper.GetUsableRev) from gclient_scm. It also removes some
references to SVN from comments in gclient_utils.
R=maruel@chromium.org
BUG=641588
Review-Url: https://chromiumcodereview.appspot.com/2393773003
Reason for revert:
Didn't help.
Original issue's description:
> gclient: kill git fetch operation that hangs.
>
> This provides env variable GCLIENT_KILL_GIT_FETCH_AFTER
> that kills git fetch if it produces no output for that
> many seconds.
>
> Note that this is not final patch, but an experiment.
> See http://crbug.com/635641#c24 for the deployment plan.
>
> BUG=635641
> R=hinoka@chromium.org
>
> Committed: f8757b7e02
TBR=hinoka@chromium.org,hinoka@google.com
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG=635641
Review-Url: https://codereview.chromium.org/2410853002
This will clean up index.lock and refs/heads/master.lock-style locks when
invoked as part of the update_scripts step on the bots.
R=dpranke@chromium.org, estaab@chromium.org, pgervais@chromium.org
BUG=421769
Review URL: https://codereview.chromium.org/1862943002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@299737 0039d316-1c4b-4281-b951-d872f2087c98
This adds the '--auto_rebase' sync option, enabling parallel updates to
automatically rebase local Git branches during sync.
BUG=None
TEST=local
Review URL: https://codereview.chromium.org/930693002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294151 0039d316-1c4b-4281-b951-d872f2087c98
Reland crrev.com/743083002, which was reverted in crrev.com/796053002
due to some test flakiness, probably related with an old version of Git on
the bots. Relanding now that the infra has been updated to Trusty (plus
adding some de-flake precautions).
Original CL Description:
Make gclient ready for the Blink (DEPS to main project) transition
This CL makes gclient understand correctly whether a git project is
being moved from DEPS to an upper project and vice-versa.
The driving use case for this is the upcoming Blink merge, where
third_party/Webkit will be removed from DEPS (and .gitignore) and will
become part of the main project.
At present state, gclient leaves the .git folder around when a project
is removed from DEPS, and that causes many problems.
Furthermore this CL solves the performance problem of bisecting across
the merge point. The subproject's (Blink) .git/ folder is moved to a
backup location (in the main checkout root) and is restored when moving
backwards, avoiding a re-fetch when bisecting across the merge point.
BUG=431469
Review URL: https://codereview.chromium.org/910913003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294082 0039d316-1c4b-4281-b951-d872f2087c98
Reason for revert:
The test intoruced is flake (see crrev.com/753543006/).
Need to look into that with more patience as it reproduces only on precise.
Original issue's description:
> Make gclient ready for the Blink (DEPS to main project) transition
>
> This CL makes gclient understand correctly whether a git project is
> being moved from DEPS to an upper project and vice-versa.
> The driving use case for this is the upcoming Blink merge, where
> third_party/Webkit will be removed from DEPS (and .gitignore) and will
> become part of the main project.
>
> At present state, gclient leaves the .git folder around when a project
> is removed from DEPS, and that causes many problems.
>
> Furthermore this CL solves the performance problem of bisecting across
> the merge point. The subproject's (Blink) .git/ folder is moved to a
> backup location (in the main checkout root) and is restored when moving
> backwards, avoiding a re-fetch when bisecting across the merge point.
>
> BUG=431469
>
> Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=293329
TBR=iannucci@chromium.org,maruel@chromium.org,jochen@chromium.org
NOTREECHECKS=true
NOTRY=true
BUG=431469
Review URL: https://codereview.chromium.org/796053002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293352 0039d316-1c4b-4281-b951-d872f2087c98
This CL makes gclient understand correctly whether a git project is
being moved from DEPS to an upper project and vice-versa.
The driving use case for this is the upcoming Blink merge, where
third_party/Webkit will be removed from DEPS (and .gitignore) and will
become part of the main project.
At present state, gclient leaves the .git folder around when a project
is removed from DEPS, and that causes many problems.
Furthermore this CL solves the performance problem of bisecting across
the merge point. The subproject's (Blink) .git/ folder is moved to a
backup location (in the main checkout root) and is restored when moving
backwards, avoiding a re-fetch when bisecting across the merge point.
BUG=431469
Review URL: https://codereview.chromium.org/743083002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293329 0039d316-1c4b-4281-b951-d872f2087c98
Then this was changed to CheckCallAndFilter it broke
the expectations of code in the buildbot tree:
scripts/slave/chromium_commands.py:extract_revisions
Alternatively we could update the expectations
of chromium_commands.py.
BUG= https://code.google.com/p/chromium/issues/detail?id=412818
Review URL: https://codereview.chromium.org/587253005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@292072 0039d316-1c4b-4281-b951-d872f2087c98
This reverts commit 931b6c6a6c.
This was causing our try bots to fail on DEPS changes.
After applying the patch successfully, apply_issue.py tries to run gclient sync to get the updated DEPS, resulting in this error:
src (ERROR)
----------------------------------------
[0:00:00] Started.
----------------------------------------
Error:
____ src at origin/master
Your index contains uncommitted changes
Please commit, stash, or reset.
BUG=413326
TESTED=Manually: https://paste.googleplex.com/6375653263802368
Review URL: https://codereview.chromium.org/563873002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@291932 0039d316-1c4b-4281-b951-d872f2087c98
Without --exit-code, diff-index always returns 0 and thus never throws
the exception for a dirty index.
R=iannucci@chromium.org
Review URL: https://codereview.chromium.org/548553003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@291881 0039d316-1c4b-4281-b951-d872f2087c98
shlex.spit(), in addition to doing str.split(' '), also strips out '\\'
from windows paths, which causes the cache_dir check to fail and Gclient to think
that we're not in cache_dir mode even if we are.
Instead of using shlex to strip data from a stdout log, I think regex is more
suited for this job.
BUG=405973
Review URL: https://codereview.chromium.org/497053002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@291446 0039d316-1c4b-4281-b951-d872f2087c98
Original CL: https://codereview.chromium.org/440263002/
Many people* have complained on chromium-dev about the long times
required to perform a full fetch over a DSL. This seems to be mostly
due to the huge size of chromium's history (~9 GB). On the other side,
not everybody is interested in downloading the full git history of
the projects. The size of git packs required to fetch a working HEAD
is one order of magnitude smaller (1.5 GB).
This change makes it possible to perform a shallow fetch (in a way
which is consistent with DEPS, leveraging git templates on clone),
reducing fetch times by 80% for those not interested in the history.
* See:
[chromium-dev] "fetch chromium" keeps hanging/getting stuck on Windows 7
[chromium-dev] Initial checkout with git taking long
[chromium-dev] Trying to get latest source code fails when fetching
[chromium-dev] Gclient sync takes too long
BUG=228996
TBR=iannucci@chromium.org,szager@chromium.org,wtc@chromium.org
Review URL: https://codereview.chromium.org/440273002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@287793 0039d316-1c4b-4281-b951-d872f2087c98
Reason for revert:
Broke the WebRTC waterfall:
http://build.chromium.org/p/tryserver.webrtc/builders/win/builds/3958/steps/gclient%20revert/logs/stdio
Original issue's description:
> Add --no-history option to fetch and gclient for shallow clones.
>
> Many people* have complained on chromium-dev about the long times
> required to perform a full fetch over a DSL. This seems to be mostly
> due to the huge size of chromium's history (~9 GB). On the other side,
> not everybody is interested in downloading the full git history of
> the projects. The size of git packs required to fetch a working HEAD
> is one order of magnitude smaller (1.5 GB).
> This change makes it possible to perform a shallow fetch (in a way
> which is consistent with DEPS, leveraging git templates on clone),
> reducing fetch times by 80% for those not interested in the history.
>
> * See:
> [chromium-dev] "fetch chromium" keeps hanging/getting stuck on Windows 7
> [chromium-dev] Initial checkout with git taking long
> [chromium-dev] Trying to get latest source code fails when fetching
> [chromium-dev] Gclient sync takes too long
>
> BUG=228996
>
> Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=287606
TBR=iannucci@chromium.org,szager@chromium.org,wtc@chromium.org
NOTREECHECKS=true
NOTRY=true
BUG=228996
Review URL: https://codereview.chromium.org/440263002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@287637 0039d316-1c4b-4281-b951-d872f2087c98
Many people* have complained on chromium-dev about the long times
required to perform a full fetch over a DSL. This seems to be mostly
due to the huge size of chromium's history (~9 GB). On the other side,
not everybody is interested in downloading the full git history of
the projects. The size of git packs required to fetch a working HEAD
is one order of magnitude smaller (1.5 GB).
This change makes it possible to perform a shallow fetch (in a way
which is consistent with DEPS, leveraging git templates on clone),
reducing fetch times by 80% for those not interested in the history.
* See:
[chromium-dev] "fetch chromium" keeps hanging/getting stuck on Windows 7
[chromium-dev] Initial checkout with git taking long
[chromium-dev] Trying to get latest source code fails when fetching
[chromium-dev] Gclient sync takes too long
BUG=228996
Review URL: https://codereview.chromium.org/437903002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@287606 0039d316-1c4b-4281-b951-d872f2087c98
Created the 'GitRefish' class to centralize 'git' refish parsing and consistent
usage by 'gclient' 'git' code.
BUG=373504
TEST=localtest
R=agable@chromium.org, iannucci@chromium.org
Review URL: https://codereview.chromium.org/328843005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@281553 0039d316-1c4b-4281-b951-d872f2087c98
Replaced multiple various invocations of 'git fetch' with calls to a common
fetch function.
BUG=373504
TEST=localtest
Review URL: https://codereview.chromium.org/327803006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@279560 0039d316-1c4b-4281-b951-d872f2087c98
This is an issue when bot_update.py is used, and a top-level
solution takes a long time to fetch. Without this change, it will
be fetched twice.
managed=False really means 'hands off my checkout', so gclient
should honor that.
R=hinoka@chromium.org,agable@chromium.org
BUG=
Review URL: https://codereview.chromium.org/334343002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@277565 0039d316-1c4b-4281-b951-d872f2087c98
This allows having a git repo cloned on a specific branch or commit, similar to
what's possible for svn solutions by specifying a url like svnrepo/branches/foo
Having the possibility to check the code out on a specific branch makes the
initial code checkout workflow simpler, and allows specifying all the relevant
info directly in the fetch recipe, instead of having to deal with branches
manually afterwards (which is the whole purpose of recipes in the first place)
So with this change, I'm able to do the following in my custom fetch recipe:
solution = {
'name' : 'src',
'url' : 'git@github.snei.sony.com:SNEI/chromium.git@refs/heads/custombranch',
<following lines skipped>
Review URL: https://codereview.chromium.org/289863012
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@273675 0039d316-1c4b-4281-b951-d872f2087c98
Edward Lemur
Andrii Shyshkalov
John Budorick
Aaron Gable
Robbie Iannucci
Daniel Cheng
Nico Weber
Paweł Hajdan, Jr
Quinten Yearsley
Mike Stipicevic
smut
maruel@chromium.org
dnj@chromium.org
primiano@chromium.org
sbc@chromium.org
mmoss@chromium.org
hinoka@chromium.org
primiano@chromium.org
smut@google.com
dnj@chromium.org
szager@chromium.org
romain.pokrzywka@gmail.com
borenet@google.com