Consider an SVN repo which was mirrored to git by git_updater. When SVN repo is migrated to github, the git histories of chromium mirror and new github one are different. As a result, hashes of the objects do not match.
Before, gclient would just get stuck at trying to fetch the repo after changing its remote url with errors like this:
...
[0:01:21] error: refs/heads/master does not point to a valid object!
[0:01:21] error: refs/remotes/origin/HEAD does not point to a valid object!
[0:01:21] error: refs/remotes/origin/master does not point to a valid object!
[0:01:21] error: refs/heads/master does not point to a valid object!
[0:01:21] error: refs/remotes/origin/HEAD does not point to a valid object!
[0:01:21] error: refs/remotes/origin/master does not point to a valid object!
[0:01:21] fatal: bad object HEAD
[0:01:21] error: /b/git-cache/chromium.googlesource.com-external-github.com-google-open--vcdiff did not send all necessary objects
The solution is to notice such state, delete the .git folder, and clone again.
BUG=523239
R=akuegel@chromium.org
Review URL: https://codereview.chromium.org/1303293002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@296417 0039d316-1c4b-4281-b951-d872f2087c98
Also removed part of the message that encourages users to disable updates to
depot_tools. If the users do not enable updates, it prevents landing changes into
project repositories that depend on having latest version of the depot_tools, e.g.
https://codereview.chromium.org/1152823005/.
R=akuegel@chromium.org
TBR=maruel@chromium.org, iannucci@chromium.org
BUG=475320
Review URL: https://codereview.chromium.org/1162153004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295486 0039d316-1c4b-4281-b951-d872f2087c98
Include links to the tracking bug and to request creating git repositories. Also
include note how to keep subversion support, if users require this.
It won't warn when git-svn clones are used.
There is no functional change except printing a warning to stderr.
Remove stale code. The CHROMIUM_SRC_URL variable is also defined in gclient.py
so it looks like a copy-paste error.
R=iannucci@chromium.org
BUG=475320
Review URL: https://codereview.chromium.org/1154173003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295478 0039d316-1c4b-4281-b951-d872f2087c98
crrev.com/348703002 has introduced some code into gclient_scm.py which
disables the .git/hooks when gclient is running in managed mode.
The disabling logic renames the individual hook files to hook.disabled
using os.rename. Conversely to what happen on Posix OSs, on Windows
os.rename does not have atomic rename semantics [1] and it fails if the
destination file already exists.
This change improves the hook disable logic.
[1] See https://bugs.python.org/issue8828 and os.replace in Python 3
BUG=474218
Review URL: https://codereview.chromium.org/1063973002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294715 0039d316-1c4b-4281-b951-d872f2087c98
This adds the '--auto_rebase' sync option, enabling parallel updates to
automatically rebase local Git branches during sync.
BUG=None
TEST=local
Review URL: https://codereview.chromium.org/930693002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294151 0039d316-1c4b-4281-b951-d872f2087c98
Reland crrev.com/743083002, which was reverted in crrev.com/796053002
due to some test flakiness, probably related with an old version of Git on
the bots. Relanding now that the infra has been updated to Trusty (plus
adding some de-flake precautions).
Original CL Description:
Make gclient ready for the Blink (DEPS to main project) transition
This CL makes gclient understand correctly whether a git project is
being moved from DEPS to an upper project and vice-versa.
The driving use case for this is the upcoming Blink merge, where
third_party/Webkit will be removed from DEPS (and .gitignore) and will
become part of the main project.
At present state, gclient leaves the .git folder around when a project
is removed from DEPS, and that causes many problems.
Furthermore this CL solves the performance problem of bisecting across
the merge point. The subproject's (Blink) .git/ folder is moved to a
backup location (in the main checkout root) and is restored when moving
backwards, avoiding a re-fetch when bisecting across the merge point.
BUG=431469
Review URL: https://codereview.chromium.org/910913003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294082 0039d316-1c4b-4281-b951-d872f2087c98
Reason for revert:
The test intoruced is flake (see crrev.com/753543006/).
Need to look into that with more patience as it reproduces only on precise.
Original issue's description:
> Make gclient ready for the Blink (DEPS to main project) transition
>
> This CL makes gclient understand correctly whether a git project is
> being moved from DEPS to an upper project and vice-versa.
> The driving use case for this is the upcoming Blink merge, where
> third_party/Webkit will be removed from DEPS (and .gitignore) and will
> become part of the main project.
>
> At present state, gclient leaves the .git folder around when a project
> is removed from DEPS, and that causes many problems.
>
> Furthermore this CL solves the performance problem of bisecting across
> the merge point. The subproject's (Blink) .git/ folder is moved to a
> backup location (in the main checkout root) and is restored when moving
> backwards, avoiding a re-fetch when bisecting across the merge point.
>
> BUG=431469
>
> Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=293329
TBR=iannucci@chromium.org,maruel@chromium.org,jochen@chromium.org
NOTREECHECKS=true
NOTRY=true
BUG=431469
Review URL: https://codereview.chromium.org/796053002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293352 0039d316-1c4b-4281-b951-d872f2087c98
This CL makes gclient understand correctly whether a git project is
being moved from DEPS to an upper project and vice-versa.
The driving use case for this is the upcoming Blink merge, where
third_party/Webkit will be removed from DEPS (and .gitignore) and will
become part of the main project.
At present state, gclient leaves the .git folder around when a project
is removed from DEPS, and that causes many problems.
Furthermore this CL solves the performance problem of bisecting across
the merge point. The subproject's (Blink) .git/ folder is moved to a
backup location (in the main checkout root) and is restored when moving
backwards, avoiding a re-fetch when bisecting across the merge point.
BUG=431469
Review URL: https://codereview.chromium.org/743083002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293329 0039d316-1c4b-4281-b951-d872f2087c98
Then this was changed to CheckCallAndFilter it broke
the expectations of code in the buildbot tree:
scripts/slave/chromium_commands.py:extract_revisions
Alternatively we could update the expectations
of chromium_commands.py.
BUG= https://code.google.com/p/chromium/issues/detail?id=412818
Review URL: https://codereview.chromium.org/587253005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@292072 0039d316-1c4b-4281-b951-d872f2087c98
This reverts commit 931b6c6a6c.
This was causing our try bots to fail on DEPS changes.
After applying the patch successfully, apply_issue.py tries to run gclient sync to get the updated DEPS, resulting in this error:
src (ERROR)
----------------------------------------
[0:00:00] Started.
----------------------------------------
Error:
____ src at origin/master
Your index contains uncommitted changes
Please commit, stash, or reset.
BUG=413326
TESTED=Manually: https://paste.googleplex.com/6375653263802368
Review URL: https://codereview.chromium.org/563873002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@291932 0039d316-1c4b-4281-b951-d872f2087c98
Without --exit-code, diff-index always returns 0 and thus never throws
the exception for a dirty index.
R=iannucci@chromium.org
Review URL: https://codereview.chromium.org/548553003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@291881 0039d316-1c4b-4281-b951-d872f2087c98
shlex.spit(), in addition to doing str.split(' '), also strips out '\\'
from windows paths, which causes the cache_dir check to fail and Gclient to think
that we're not in cache_dir mode even if we are.
Instead of using shlex to strip data from a stdout log, I think regex is more
suited for this job.
BUG=405973
Review URL: https://codereview.chromium.org/497053002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@291446 0039d316-1c4b-4281-b951-d872f2087c98
This allows cache_dir to be enabled on bots when the individual repo path has
changes (like android_tools, which has a bunch of generated, non-ignored .h
files)
R=agable@chromium.org, szager@chromium.org
BUG=
Review URL: https://codereview.chromium.org/488273003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@290943 0039d316-1c4b-4281-b951-d872f2087c98
Original CL: https://codereview.chromium.org/440263002/
Many people* have complained on chromium-dev about the long times
required to perform a full fetch over a DSL. This seems to be mostly
due to the huge size of chromium's history (~9 GB). On the other side,
not everybody is interested in downloading the full git history of
the projects. The size of git packs required to fetch a working HEAD
is one order of magnitude smaller (1.5 GB).
This change makes it possible to perform a shallow fetch (in a way
which is consistent with DEPS, leveraging git templates on clone),
reducing fetch times by 80% for those not interested in the history.
* See:
[chromium-dev] "fetch chromium" keeps hanging/getting stuck on Windows 7
[chromium-dev] Initial checkout with git taking long
[chromium-dev] Trying to get latest source code fails when fetching
[chromium-dev] Gclient sync takes too long
BUG=228996
TBR=iannucci@chromium.org,szager@chromium.org,wtc@chromium.org
Review URL: https://codereview.chromium.org/440273002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@287793 0039d316-1c4b-4281-b951-d872f2087c98
Reason for revert:
Broke the WebRTC waterfall:
http://build.chromium.org/p/tryserver.webrtc/builders/win/builds/3958/steps/gclient%20revert/logs/stdio
Original issue's description:
> Add --no-history option to fetch and gclient for shallow clones.
>
> Many people* have complained on chromium-dev about the long times
> required to perform a full fetch over a DSL. This seems to be mostly
> due to the huge size of chromium's history (~9 GB). On the other side,
> not everybody is interested in downloading the full git history of
> the projects. The size of git packs required to fetch a working HEAD
> is one order of magnitude smaller (1.5 GB).
> This change makes it possible to perform a shallow fetch (in a way
> which is consistent with DEPS, leveraging git templates on clone),
> reducing fetch times by 80% for those not interested in the history.
>
> * See:
> [chromium-dev] "fetch chromium" keeps hanging/getting stuck on Windows 7
> [chromium-dev] Initial checkout with git taking long
> [chromium-dev] Trying to get latest source code fails when fetching
> [chromium-dev] Gclient sync takes too long
>
> BUG=228996
>
> Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=287606
TBR=iannucci@chromium.org,szager@chromium.org,wtc@chromium.org
NOTREECHECKS=true
NOTRY=true
BUG=228996
Review URL: https://codereview.chromium.org/440263002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@287637 0039d316-1c4b-4281-b951-d872f2087c98
Many people* have complained on chromium-dev about the long times
required to perform a full fetch over a DSL. This seems to be mostly
due to the huge size of chromium's history (~9 GB). On the other side,
not everybody is interested in downloading the full git history of
the projects. The size of git packs required to fetch a working HEAD
is one order of magnitude smaller (1.5 GB).
This change makes it possible to perform a shallow fetch (in a way
which is consistent with DEPS, leveraging git templates on clone),
reducing fetch times by 80% for those not interested in the history.
* See:
[chromium-dev] "fetch chromium" keeps hanging/getting stuck on Windows 7
[chromium-dev] Initial checkout with git taking long
[chromium-dev] Trying to get latest source code fails when fetching
[chromium-dev] Gclient sync takes too long
BUG=228996
Review URL: https://codereview.chromium.org/437903002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@287606 0039d316-1c4b-4281-b951-d872f2087c98
Created the 'GitRefish' class to centralize 'git' refish parsing and consistent
usage by 'gclient' 'git' code.
BUG=373504
TEST=localtest
R=agable@chromium.org, iannucci@chromium.org
Review URL: https://codereview.chromium.org/328843005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@281553 0039d316-1c4b-4281-b951-d872f2087c98
Replaced multiple various invocations of 'git fetch' with calls to a common
fetch function.
BUG=373504
TEST=localtest
Review URL: https://codereview.chromium.org/327803006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@279560 0039d316-1c4b-4281-b951-d872f2087c98
Replaced multiple various invocations of 'git checkout' with calls to a common
checkout function.
BUG=373504
TEST=localtest
R=agable@chromium.org, iannucci@chromium.org
Review URL: https://codereview.chromium.org/326153003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@279207 0039d316-1c4b-4281-b951-d872f2087c98
Do it only when both --force and --delete_unversioned_trees are specified.
BUG=386878
R=iannucci@chromium.org
Review URL: https://codereview.chromium.org/334633004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@278770 0039d316-1c4b-4281-b951-d872f2087c98
This is an issue when bot_update.py is used, and a top-level
solution takes a long time to fetch. Without this change, it will
be fetched twice.
managed=False really means 'hands off my checkout', so gclient
should honor that.
R=hinoka@chromium.org,agable@chromium.org
BUG=
Review URL: https://codereview.chromium.org/334343002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@277565 0039d316-1c4b-4281-b951-d872f2087c98
This will avoid errors where _Clone is called for a new DEP, but where the DEP
refers to a branch commit, thus causing the clone to fail because it can't
checkout that commit (such as happens currently with 'fetch v8').
Review URL: https://codereview.chromium.org/307223006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@274347 0039d316-1c4b-4281-b951-d872f2087c98
This allows having a git repo cloned on a specific branch or commit, similar to
what's possible for svn solutions by specifying a url like svnrepo/branches/foo
Having the possibility to check the code out on a specific branch makes the
initial code checkout workflow simpler, and allows specifying all the relevant
info directly in the fetch recipe, instead of having to deal with branches
manually afterwards (which is the whole purpose of recipes in the first place)
So with this change, I'm able to do the following in my custom fetch recipe:
solution = {
'name' : 'src',
'url' : 'git@github.snei.sony.com:SNEI/chromium.git@refs/heads/custombranch',
<following lines skipped>
Review URL: https://codereview.chromium.org/289863012
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@273675 0039d316-1c4b-4281-b951-d872f2087c98
This is a special case where we want to run a git command, even if
it's an SVNWrapper instance.
R=iannucci@chromium.org,agable@chromium.org
BUG=369145
Review URL: https://codereview.chromium.org/264913006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@267707 0039d316-1c4b-4281-b951-d872f2087c98
Currently, any bot run with "--revision=project@origin/master" fails, because
it fetches the remote ref into refs/remotes/origin/master, but then checks out
refs/heads/master, which hasn't been rolled forward. This is very similar to
the bug fixed by https://codereview.chromium.org/85473007
However, I am very wary of this patch, because I am worried about changing
something commented with " For compatibility...". Marc-Antoine, please comment.
R=maruel@chromium.org, smut@chromium.org, stip@chromium.org
Review URL: https://codereview.chromium.org/244253007
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@266251 0039d316-1c4b-4281-b951-d872f2087c98
This isn't doing anything, and is currently costing 20 seconds of idle time
per bot_update run. Lets remove it until we convert lkcr/lkgr to tags.
BUG=261741
Review URL: https://codereview.chromium.org/256683002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@266055 0039d316-1c4b-4281-b951-d872f2087c98
--with_branch_heads doesn't work with cache_dir, this fixes it.
BUG=
Review URL: https://codereview.chromium.org/239083007
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@264331 0039d316-1c4b-4281-b951-d872f2087c98
Evidence indicates that running non-builtin git commands is very
slow in msysgit, slow enough to dominate the running time of
gclient sync. With this change, gclient never shells out to
git-cache; it import the lib directly instead.
R=agable@chromium.org,hinoka@chromium.org
BUG=
Review URL: https://codereview.chromium.org/229653002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@262759 0039d316-1c4b-4281-b951-d872f2087c98
Features:
- Non-verbose output is now limited to a one-line progress
indicator.
- Verbose output is now collated per subprocess. As soon as a
subprocess finishes, its full output is dumped to terminal.
- Verbose output is prefixed with timestamps representing elapsed
time since the beginning of the gclient invocation.
- git progress indicators ("Receiving objects", etc.) are limited to
one line every 10 seconds.
- In both verbose and non-verbose mode, if a failure occurs, the
full output of the failed update operation is dumped to terminal
just before exit.
- In the event that updates are progressing, but slowly,
"Still working" messages will be printed periodically, to pacify
users and buildbots.
BUG=
R=hinoka@google.com
Review URL: https://codereview.chromium.org/227163002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@262500 0039d316-1c4b-4281-b951-d872f2087c98
This causes git commands to be completely silent (they don't even print
______ running 'git foo bar') if the git command itself is completely silent.
This mirrors the behavior of SVN commands. This is useful for commands like
'gclient diff', which should print no output at all if there is no diff.
R=szager@chromium.org, vadimsh@chromium.org
Review URL: https://codereview.chromium.org/218993009
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@260758 0039d316-1c4b-4281-b951-d872f2087c98
We don't want to do a shallow clone into the cache because it is not supported
on versions of git older than 1.9.
BUG=261741
Review URL: https://codereview.chromium.org/198103012
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@257578 0039d316-1c4b-4281-b951-d872f2087c98
Reason for revert:
Broke `gclient sync` for me, failing with:
Error: 1> Can't update/checkout /Volumes/MacintoshHD2/src/chrome-git/src if an unversioned directory is present. Delete the directory and try again.
For someone else, it broke it with:
% gclient sync
________ unmanaged solution; skipping src
Error: Command svn info --xml returned non-zero exit status 1 in /Users/pawliger/chromium/src/.
<?xml version="1.0" encoding="UTF-8"?>
<info>
svn: E155007: '/Users/pawliger/chromium/src' is not a working copy
Original issue's description:
> Another attempt: gclient: delete mismatching checkouts
>
> Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=255930
TBR=iannucci@chromium.org,szager@chromium.org,maruel@chromium.org,mmoss@chromium.org,borenet@google.com
NOTREECHECKS=true
NOTRY=true
Review URL: https://codereview.chromium.org/192323006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@256005 0039d316-1c4b-4281-b951-d872f2087c98
We don't want to run it in the DEPS directory, which is apperently what
the default behavior is.
BUG=339171
Review URL: https://codereview.chromium.org/184793003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@254266 0039d316-1c4b-4281-b951-d872f2087c98
Instead of having custom logic for dealing with cache directories, use
git_cache.py to populate caches.
Also fixes a bug in git_cache.py where it was looking for lockfiles in cwd rather than the cache dir.
Other changes:
* _Run now returns output.
* Always print to stdout in CheckCallAndFilterOutput, even if it gets a carriage return. This is done because git progress report are carriage returns and not newlines and we don't want everything on the same line and not strip out the CRs.
* Removed members changed tests, its not very useful to know a new import is added.
BUG=339171
Review URL: https://codereview.chromium.org/180243006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@254248 0039d316-1c4b-4281-b951-d872f2087c98
This reverts commit 6ff5448534.
'fetch --nohooks chromium' is broken with this change.
BUG=
Review URL: https://codereview.chromium.org/177003023
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@253640 0039d316-1c4b-4281-b951-d872f2087c98
The git cache command is a central place to manage a machine's git cache.
It provides two subcommands:
* populate -- creates or updates the cache of a given repository
* exists -- prints the path to the cache of a repo, if it exists
Gclient, deps2git, bot_update, and any other tools that touch the cache will
be able to use this command to make sure that everyone is interacting with
the cache in the same way.
R=hinoka@chromium.org, iannucci@chromium.org
BUG=339168
Review URL: https://codereview.chromium.org/164823002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@253007 0039d316-1c4b-4281-b951-d872f2087c98
One line for the fix, 30 lines for the test :)
R=kbr@chromium.org, mmoss@chromium.org, stip@chromium.org, szager@chromium.org
BUG=345073
Review URL: https://codereview.chromium.org/165443004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@252138 0039d316-1c4b-4281-b951-d872f2087c98
Reason for revert:
This breaks `gclient sync` for me. Before this patch:
________ found .git directory; skipping src
After this patch:
Error: 1> Can't update/checkout /Volumes/MacintoshHD2/src/chrome-git/src if an unversioned directory is present. Delete the directory and try again.
Original issue's description:
> If the destination directory doesn't contain the desired repo, delete it
>
> Basic functionality - this may not be in quite the right place.
>
> Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=245404
TBR=iannucci@chromium.org,ilevy@chromium.org,cmp@chromium.org,maruel@chromium.org,morrita@chromium.org,borenet@google.com
NOTREECHECKS=true
NOTRY=true
Review URL: https://codereview.chromium.org/141633005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@245530 0039d316-1c4b-4281-b951-d872f2087c98
Basic functionality - this may not be in quite the right place.
Review URL: https://codereview.chromium.org/61623008
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@245404 0039d316-1c4b-4281-b951-d872f2087c98
This is a serious bug, if a version is unpinned, it would become stuck at
whatever a local branch 'master' may point to. This is extremely bad when a
pinned dependency is unpinned on purpose, for example on a canary master. We
reproduced this problem on the chromium.swarm master by setting a custom_vars
swarming_revision to '', which then stopped syncing at the expected
origin/master commit but instead got stuck to what happened to be a local master
branch.
This happens on git dependencies if the main solution is using svn. This doesn't
happen on git checkouts using .DEPS.git due to bug 323233.
The issue looks like:
________ running 'git reset --hard HEAD' in '/path/src/tools/swarming_client'
HEAD is now at 4727bd5 Some commit
Checked out revision d908a546e28d1e9f85f5690cf6c3a080f06ba711
Note that HEAD and what is checked out do not match.
Also reduce the hardcoding of 'origin' by creating a variable named 'remote'.
R=petermayo@chromium.org
BUG=322961
Review URL: https://codereview.chromium.org/85473007
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@237332 0039d316-1c4b-4281-b951-d872f2087c98
This helps prevent timeout when checking out large repositories (like blink)
using the git cache_dir.
R=iannucci@chromium.org, maruel@chromium.org
BUG=295119
Review URL: https://codereview.chromium.org/43273005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@231111 0039d316-1c4b-4281-b951-d872f2087c98
Add retry for all git operations that speak to a remote. This
should smooth out issues with the git/gerrit-on-borg service.
Review URL: https://codereview.chromium.org/26234004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@229219 0039d316-1c4b-4281-b951-d872f2087c98
This fixes an error if the branch already exists.
Review URL: https://chromiumcodereview.appspot.com/24360009
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@225278 0039d316-1c4b-4281-b951-d872f2087c98
This is in order to support 'features' like got_revision for build systems
unwilling to scrape the human-readable output of gclient in order to extract
basic information :)
R=agable@chromium.org, maruel@chromium.org
BUG=265618
Review URL: https://chromiumcodereview.appspot.com/23753008
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@225046 0039d316-1c4b-4281-b951-d872f2087c98
This solves a problem with "os.rename" sometimes failing with an
exception after cloning a dependency to a temporary directory. It's
possible the dying git processes still hold a handle to the directory.
Since gclient delete the temporary directory regardless of the success
of the process, it results in a lot of GB downloaded for nothing.
SafeRename solves this by retrying a few times if the renaming fails,
sleeping one second every time to get other processes the time to
release their lock on the directory. It gives up retrying after 15 times.
BUG=
R=maruel@chromium.org
Review URL: https://chromiumcodereview.appspot.com/23875041
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@224372 0039d316-1c4b-4281-b951-d872f2087c98
Add an option in DEPS files to clone a project into a temp dir
and then copy into expected final dir. This allows checking out a
git repo into a folder which is non-empty. It is useful for projects
that are embedded in src/ but want to specify the revision of
src/ in the embedded project (such as android private).
BUG=165280
Review URL: https://chromiumcodereview.appspot.com/19359002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@212720 0039d316-1c4b-4281-b951-d872f2087c98
Also open the altfile with 'w' because we assume ownership over the object
alternates. The logic to correctly support other alternates would be quite
complex, so don't even imply support.
R=maruel@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/19777004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@212426 0039d316-1c4b-4281-b951-d872f2087c98
This caused problems for repo-based projects that need to install the
standard Change-Id commit-msg hook.
With the commit-msg hook gone, we can also reinstate the --template
argument to 'git clone'.
BUG=235215
R=iannucci@chromium.org, ilevy@chromium.org, maruel@chromium.org
Review URL: https://codereview.chromium.org/18083033
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@211464 0039d316-1c4b-4281-b951-d872f2087c98
Forcing function for developers to create a .netrc file.
BUG=
Review URL: https://codereview.chromium.org/18949002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@210686 0039d316-1c4b-4281-b951-d872f2087c98
timer, i.e. ~900 seconds.
This should at least help make 'fetch chromium' slightly more reliable
over somewhat slow connections.
BUG=230691,247173
R=rogerta@chromium.org
Review URL: https://codereview.chromium.org/18150002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@210190 0039d316-1c4b-4281-b951-d872f2087c98
This enables significant time savings, especially since file_list only exists to
enable file-specific hooks (which, AFAIK, nothing actually uses). On a z620
(linux) using the cached git repos, a first-time `gclient sync --nohooks` takes:
* (with) 131.06s user 14.10s system 117% cpu 2:03.89 total
* (without) 482.13s user 189.35s system 144% cpu 7:45.63 total
This change makes nohooks cause file_list to be None if we don't need to
accumulate it, and updates GitWrapper and SvnWrapper appropriately.
R=szager@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/18541006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@210026 0039d316-1c4b-4281-b951-d872f2087c98
Instead of cloning straight into place, clones are made to a global cache dir,
and then local (using --shared) clones are made from the cache to the final
resting place. This means the 'final' clones are full repos with no shenanigans,
meaning that branches, commits, etc. all work, which should allow the rest of
the gclient ecosystem to work without change as well.
The primary benefit is, of course, reduced network IO, and a much lower cost for
'clobber' operations (assuming we don't clobber the cache). It also means that
a given bot can have a greater number of checkouts, since the entire git history
will only be stored once per machine, instead of once per checkout.
R=dpranke@chromium.org, szager@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/18328003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@210024 0039d316-1c4b-4281-b951-d872f2087c98
- Use config --local for properties that should be local.
- Remove git fetch retry logic, it should not be necessary
with the googlesource hosts.
- Include .strip() in GIT.Capture, simplifying other calls.
- Safer testing of whether dep revision is a sha.
- Fix refs/remotes/ DEPS branch detection
Review URL: https://chromiumcodereview.appspot.com/18262002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@209825 0039d316-1c4b-4281-b951-d872f2087c98
This will be used on the bots when calling apply_patch, which commits to git
repos.
R=dpranke@chromium.org,szager@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/18006002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@209040 0039d316-1c4b-4281-b951-d872f2087c98
As noticed in the TODO, raw_input (used by ask_for_data) works badly
on background threads. On my Windows it doesn't even print the prompt which
makes it hard to interact with it.
Better to just abort and tell the user of gclient how to proceed.
BUG=
Review URL: https://chromiumcodereview.appspot.com/15421005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@201295 0039d316-1c4b-4281-b951-d872f2087c98
Allow users to set a git config entry which stops
gclient from messing with remote.origin.url. Useful
for using experimental repos.
Review URL: https://chromiumcodereview.appspot.com/14731015
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@199532 0039d316-1c4b-4281-b951-d872f2087c98
The 'RemoveDirectory()' function in gclient_utils is deprecated and
rmtree() should be used instead for consistency.
This patch modifies all clients in depot_tools to use rmtree() instead
and removes the RemoveDirectory function.
+ The SVNWrapperTestCase.testRevertNoDotSvn() mocking
expectation has been slightly changed. This was required
because the test invokes code that used to call
gclient_utils.RemoveDirectory() directly, while only
gclient_utils.rmtree() was mocked.
BUG=NONE
R=maruel@chromium.org, ilevy@chromium.org
TEST=manually run gclient_utils_test / gclient_smoketest / scm_unittest / gclient_scm_test
Review URL: https://chromiumcodereview.appspot.com/14134010
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@196133 0039d316-1c4b-4281-b951-d872f2087c98
This is a bit of future-proofing. The git hooks in depot_tools/git-templates/hooks all try to forward to build/git-hooks.
Currently, the only actual hook we have is in src/build/git-hooks/pre-commit, which (usefully) makes sure no one inadvertently commits a submodule change. If we want to add more git hooks, we can simply commit them to build/git-hooks, rather than asking everyone to modify .git/hooks in their checkout.
NOTRY=true
Review URL: https://chromiumcodereview.appspot.com/14230012
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@195336 0039d316-1c4b-4281-b951-d872f2087c98
This allows the branch-heads to be pulled even if not configured on the
original clone.
Review URL: https://codereview.chromium.org/13951009
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@194382 0039d316-1c4b-4281-b951-d872f2087c98
This was causing extremely slow gclient sync's, and was preventing gclient from
outputting anything if a slow submodule needed significant updating.
R=cmp@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/13702002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@192475 0039d316-1c4b-4281-b951-d872f2087c98
This reverts commit 6a2e4710bf.
cmp says:
This CL causes NewGit and Git submodules workflows to comingle in ways that result in bad and unexpected failures for new and existing users. One failure is for the new Blink developers who need to use git-svn with their WebKit third_party checkout. Another failure is in the Windows NewGit workflow where gclient sync takes longer than necessary to complete.
The right approach at this point given the costs of this change is to revert it and rethink the best way to get projects the functionality they need while restoring the functionality existing projects have required for over a year now.
Please discuss the recursive/recurseSubmodules code face-to-face with cmp@google.com before attempting to make this live again. Thanks.
R=cmp@chromium.org
BUG=224074
Review URL: https://chromiumcodereview.appspot.com/13470027
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@192236 0039d316-1c4b-4281-b951-d872f2087c98
This causes the 'git config' to only add or update the branch-heads fetch
entry, and not replace any other existing entry (e.g. refs/heads/*).
Review URL: https://chromiumcodereview.appspot.com/12634013
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@189326 0039d316-1c4b-4281-b951-d872f2087c98
Some versions of git don't support 'clone --config'.
Review URL: https://codereview.chromium.org/12688005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@187296 0039d316-1c4b-4281-b951-d872f2087c98
Some SVN-only trybots get errors when running gclient revert with this
change. I assume it's because the flag is not present for the gclient
revert command, and that the code checking if options.with_branch_heads
is set is common to both revert and sync.
BUG=none
TEST=reverted locally and gclient revert worked again.
Review URL: https://chromiumcodereview.appspot.com/12616006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@186967 0039d316-1c4b-4281-b951-d872f2087c98
The "branch-heads" are refspecs created from upstream svn branches. This flag
is mostly needed for official builders, and adds about 1/2GB of data to the
checkout, so not currently enabling by default.
Review URL: https://codereview.chromium.org/12546017
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@186598 0039d316-1c4b-4281-b951-d872f2087c98
Currently, if an unversioned directory is present where we would expect
a versioned repository to be, the following error is printed:
Can't update/checkout %s if an unversioned directory is present.
Delete the directory and try again.
If --reset and --delete_unversioned_trees are used, gclient should delete
the unversioned directory in this case.
This problem can be reproduced using the following recipe:
$ rm -rf src/third_party/webrtc/.svn
$ gclient sync -nRftD
BUG=none
TEST=Verify that above error is fixed. Run all smoke tests.
Review URL: https://chromiumcodereview.appspot.com/11366239
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@167942 0039d316-1c4b-4281-b951-d872f2087c98
Takes into account the new submodule-enabled repository layout.
Review URL: https://codereview.chromium.org/11098064
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@161403 0039d316-1c4b-4281-b951-d872f2087c98
The try slaves run 'git revert' to clean up from the previous run. This only runs 'git reset --hard', which does not remove any untracked files, so if a CL adds a new file it will still be present after 'gclient revert'. Adding git clean -f -d removes untracked files (-f) and directories (-d).
Review URL: https://chromiumcodereview.appspot.com/10986032
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@159020 0039d316-1c4b-4281-b951-d872f2087c98
This should make a `gclient sync` faster, especially on Windows, where git hangs regularly.
BUG=none
TEST=gclient sync -v doesn't print "Fetching origin" all the time
Review URL: https://chromiumcodereview.appspot.com/10668020
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@143898 0039d316-1c4b-4281-b951-d872f2087c98
The effect of this is to preserve the behavior of `git status`,
i.e., only changes in the top-level repository will be reported.
Updated test expectations; I have no idea what I'm doing.
Review URL: https://chromiumcodereview.appspot.com/10454088
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@140396 0039d316-1c4b-4281-b951-d872f2087c98
into stdout and creating the GitDiffFilterer for git repository
BUG=125894
TEST=
Review URL: http://codereview.chromium.org/10317002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@135324 0039d316-1c4b-4281-b951-d872f2087c98
The first approach crrev.com/132446 was reverted due to
Windows build breaks. This approach is more robust because
it actually tests the binary by running the command actually.
BUG=114483
TEST=succeeded on Linux
Review URL: http://codereview.chromium.org/10103024
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@132783 0039d316-1c4b-4281-b951-d872f2087c98
iannucci@chromium.org
tandrii@chromium.org
mmoss@chromium.org
szager@chromium.org
sergiyb@chromium.org
maruel@chromium.org
primiano@chromium.org
e.hakkinen@samsung.com
dnj@chromium.org
hinoka@chromium.org
avakulenko@google.com
agable@chromium.org
sbc@chromium.org
smut@google.com
vadimsh@chromium.org
szager@chromium.org
hinoka@chromium.org
iannucci@chromium.org
primiano@chromium.org
smut@google.com
dnj@chromium.org
mmoss@chromium.org
romain.pokrzywka@gmail.com
borenet@google.com
jochen@chromium.org
agable@chromium.org
hinoka@google.com
stip@chromium.org
thakis@chromium.org
zty@chromium.org
mariakhomenko@chromium.org
bauerb@chromium.org
maruel@chromium.org
scottmg@chromium.org
cyrille@nnamrak.org
kustermann@google.com
ilevy@chromium.org
joi@chromium.org
xusydoc@chromium.org
bratell@opera.com
digit@chromium.org
jam@chromium.org
kjellander@chromium.org
davidjames@chromium.org
szager@google.com
lliabraa@chromium.org
haitao.feng@intel.com
mukai@chromium.org