This pins gsutil to a vanilla 4.7 instead of the weird custom 3.4 we have in depot_tools
BUG=
R=pgervais@chromium.org
Review URL: https://codereview.chromium.org/797663003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293413 0039d316-1c4b-4281-b951-d872f2087c98
So the original intention of moving it to a different directory to be deleted later
was to (1) save it for diagnosis (2) be a single inode swap rather than a long
rmtree delete.
1. No one is actually looking at these directories
2. The tmpdir sometimes end up on a different partition, so it ends up being
a copy + delete instead. Since we're not getting timeouts from that, its
probably actually better to just straight up delete it.
BUG=410727
Review URL: https://codereview.chromium.org/538993002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@291818 0039d316-1c4b-4281-b951-d872f2087c98
Seems like some combination of things (bad cache dir, shallow checkout, etc) causes
git cache to not clean up properly before trying to move new directory A into
already existing directory B.
So the logic is all there to detect that the cache is invalid, its just not
being cleaned up properly during shallow checkouts.
BUG=406864
TBR=iannucci
Review URL: https://codereview.chromium.org/499123002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@291587 0039d316-1c4b-4281-b951-d872f2087c98
Some options have words separated by underscores. Added options with
same name and underscores replaced by hyphens.
BUG=400953
Review URL: https://codereview.chromium.org/436963005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@288366 0039d316-1c4b-4281-b951-d872f2087c98
If you're in a git checkout cloned from the git cache, this will:
- Update the cache with the latest upstream commits.
- Update the cwd with the latest commits from the cache.
For example:
> cd $HOME/workspace/chromium/src
> git cache fetch
R=agable@chromium.org,iannucci@chromium.org
BUG=
Review URL: https://codereview.chromium.org/440213005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@288140 0039d316-1c4b-4281-b951-d872f2087c98
Git cache sometimes fail on:
Traceback (most recent call last):
File "E:\b\depot_tools\\gclient.py", line 2064, in <module>
sys.exit(Main(sys.argv[1:]))
File "E:\b\depot_tools\\gclient.py", line 2052, in Main
return dispatcher.execute(OptionParser(), argv)
File "E:\b\depot_tools\subcommand.py", line 245, in execute
return command(parser, args[1:])
File "E:\b\depot_tools\\gclient.py", line 1830, in CMDsync
ret = client.RunOnDeps('update', args)
File "E:\b\depot_tools\\gclient.py", line 1342, in RunOnDeps
work_queue.flush(revision_overrides, command, args, options=self._options)
File "E:\b\depot_tools\gclient_utils.py", line 852, in flush
self._run_one_task(self.queued.pop(i), args, kwargs)
File "E:\b\depot_tools\gclient_utils.py", line 944, in _run_one_task
task_item.run(*args, **kwargs)
File "E:\b\depot_tools\\gclient.py", line 744, in run
file_list)
File "E:\b\depot_tools\gclient_scm.py", line 160, in RunCommand
return getattr(self, command)(options, args, file_list)
File "E:\b\depot_tools\gclient_scm.py", line 387, in update
self._UpdateMirror(mirror, options)
File "E:\b\depot_tools\gclient_scm.py", line 802, in _UpdateMirror
ignore_lock=options.ignore_locks)
File "E:\b\depot_tools\git_cache.py", line 409, in populate
os.rename(tempdir, self.mirror_path)
WindowsError: [Error 183] Cannot create a file when that file already exists
It would appear that its being racy, but otherwise it doesn't make any sense. A theory
is that this could be running twice and stepping on each other. Allowing this
to pass on OSError allows us to test this theory.
BUG=395333
Review URL: https://codereview.chromium.org/408653002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@284272 0039d316-1c4b-4281-b951-d872f2087c98
We're seeing fetches fail in interesting ways:
running "git fetch -v --progress origin +refs/heads/*:refs/heads/*" in "/mnt/scratch0/b_used/build/slave/cache_dir/chrome--internal.googlesource.com-chrome-src--internal"
error: object file ./objects/a1/4bd89aa4cc7d7bbad7594cba0ae73e99dbb54c is empty
error: object file ./objects/a1/4bd89aa4cc7d7bbad7594cba0ae73e99dbb54c is empty
fatal: loose object a14bd89aa4cc7d7bbad7594cba0ae73e99dbb54c (stored in ./objects/a1/4bd89aa4cc7d7bbad7594cba0ae73e99dbb54c) is corrupt
fatal: The remote end hung up unexpectedly
And then the cache becomes corrupted. This blows the cache away if this happens.
BUG=261741
Review URL: https://codereview.chromium.org/352543003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@280123 0039d316-1c4b-4281-b951-d872f2087c98
Unfortunately, the locking mechanism is still flaky on Windows.
bot_update.py will use this, since we can be certain that there
won't be overlapping access to the cache.
R=hinoka@google.com, agable@chromium.org, hinoka@chromium.org
BUG=
Review URL: https://codereview.chromium.org/340953003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@278490 0039d316-1c4b-4281-b951-d872f2087c98
A number of bots have been running out of disk space because they
are accumulating temp dirs and temp pack files. Be more careful
to clean up temp dirs whenever UnlockAll() is called.
R=hinoka@chromium.org,agable@chromium.org
BUG=
Review URL: https://codereview.chromium.org/335253002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@277812 0039d316-1c4b-4281-b951-d872f2087c98
When a git-cache operation is interrupted, it can leave behind large
temporary pack files. Over time, these pack files will fill up
disks.
R=agable@chromium.org,hinoka@chromium.org,iannucci@chromium.org
BUG=352268
Review URL: https://codereview.chromium.org/326203003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@276208 0039d316-1c4b-4281-b951-d872f2087c98
Last patch (https://codereview.chromium.org/313933005/) forgot to take into account using a boto file (setting boto to os.devnull
means connect unauthenticated, setting it to None will allow it to do the default
behavior)
This patch will always use a boto file, which should be fine.
BUG=261741
Review URL: https://codereview.chromium.org/314203002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@275230 0039d316-1c4b-4281-b951-d872f2087c98
This is need, e.g., to suppress logging message for 'exists'.
TBR=agable@chromium.org,hinoka@chromium.org
BUG=
Review URL: https://codereview.chromium.org/299163002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@272450 0039d316-1c4b-4281-b951-d872f2087c98
This is because the version of unzip on osx doesn't support zip64, so it can't
unzip large repositories like blink.
BUG=261741
Review URL: https://codereview.chromium.org/251413002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@266151 0039d316-1c4b-4281-b951-d872f2087c98
Sometimes, removing lockfiles fails on windows, for obscure reasons.
Following advice from gclient_utils.rmtree, lock file removal has
been improved by using the builtin executable 'del' and retrying 3
times in case it fails.
BUG=
Review URL: https://codereview.chromium.org/240653002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@264557 0039d316-1c4b-4281-b951-d872f2087c98
Evidence indicates that running non-builtin git commands is very
slow in msysgit, slow enough to dominate the running time of
gclient sync. With this change, gclient never shells out to
git-cache; it import the lib directly instead.
R=agable@chromium.org,hinoka@chromium.org
BUG=
Review URL: https://codereview.chromium.org/229653002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@262759 0039d316-1c4b-4281-b951-d872f2087c98
This is based off the svn tarball bootstrap script in gclient_scm.py
Also included is a new command to create a new zarchive/zipball.
BUG=261741
Review URL: https://codereview.chromium.org/220423002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@261002 0039d316-1c4b-4281-b951-d872f2087c98
git_cache rewrites all urls to https://netloc/path when
setting them as remote urls. This doesn't work when the
remote is actually local on disk, and the "url" is a
/path/to/repo. This seems like an unlikely case, but it
is leveraged by testing, and it doesn't hurt to support
it, so now we do.
R=iannucci@chromium.org
BUG=261619
Review URL: https://codereview.chromium.org/196493002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@256386 0039d316-1c4b-4281-b951-d872f2087c98
There is an issue where:
1. Lock for asdf.git acquires, which creates asdf.lock
2. asdf.git is cloned into tmp_asdf.git
3. Buildbot crashes, because timeouts
4. neither asdf.lock or tmp_asdf.git/config.lock gets cleaned out on the next unlock.
This aims to fix this issue by aggressively unlocking everything it can find that
vaguely resembles a lock.
BUG=339171
Review URL: https://codereview.chromium.org/187283005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@255138 0039d316-1c4b-4281-b951-d872f2087c98
Instead of having custom logic for dealing with cache directories, use
git_cache.py to populate caches.
Also fixes a bug in git_cache.py where it was looking for lockfiles in cwd rather than the cache dir.
Other changes:
* _Run now returns output.
* Always print to stdout in CheckCallAndFilterOutput, even if it gets a carriage return. This is done because git progress report are carriage returns and not newlines and we don't want everything on the same line and not strip out the CRs.
* Removed members changed tests, its not very useful to know a new import is added.
BUG=339171
Review URL: https://codereview.chromium.org/180243006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@254248 0039d316-1c4b-4281-b951-d872f2087c98
For some reason "--tags" wasn't working on some versions of git (pre-1.9)
BUG=339171
Review URL: https://codereview.chromium.org/183813004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@253929 0039d316-1c4b-4281-b951-d872f2087c98
The git cache command is a central place to manage a machine's git cache.
It provides two subcommands:
* populate -- creates or updates the cache of a given repository
* exists -- prints the path to the cache of a repo, if it exists
Gclient, deps2git, bot_update, and any other tools that touch the cache will
be able to use this command to make sure that everyone is interacting with
the cache in the same way.
R=hinoka@chromium.org, iannucci@chromium.org
BUG=339168
Review URL: https://codereview.chromium.org/164823002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@253007 0039d316-1c4b-4281-b951-d872f2087c98
hinoka@chromium.org
szager@chromium.org
pgervais@chromium.org
hinoka@chromium.org
hinoka@google.com
agable@chromium.org