Paths containing ':' characters are illegal only on Windows platforms.
So, ssh_url like git@github.com:chromium/chromium.git will generate
runtime exception on Windows. We replace ':' with '__' to avoid
exceptions in the ssh form of git_url case.
R=ddoman@chromium.org
Bug: 329781388
Change-Id: Ia4db6a0927b40b521477e5d90addb4056c2c8ef8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5369296
Auto-Submit: 吴金立 <wujinli.cn@gmail.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
git_cache populate can get interrupted midway (e.g. by LUCI CV sending a
signal that build is no longer needed). When that happens, a git mirror
may be in a state where some commits are available, but cloning such
repositry results in an empty repsitory.
This change ensures that the initial fetch operation finished
successfully.
R=ddoman@google.com
Bug: 1517944
Change-Id: I1ee860860877dbfff7a444b45fe4515fe26b248c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5219032
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Scott Lee <ddoman@chromium.org>
Reformat this dir by itself to help merging with conflicts with other CLs.
Reformatted using:
parallel ./yapf -i -- tests/*.py
~/chromiumos/chromite/contrib/reflow_overlong_comments tests/*.py
These files still had lines (strings) that were too long, so the pylint
warnings were suppressed with a TODO.
tests/bot_update_coverage_test.py
tests/cipd_bootstrap_test.py
tests/gclient_eval_unittest.py
tests/gclient_git_smoketest.py
tests/gclient_scm_test.py
tests/gclient_smoketest.py
tests/gclient_test.py
tests/gclient_transitions_smoketest.py
tests/gclient_utils_test.py
tests/git_cl_test.py
tests/git_hyper_blame_test.py
tests/git_rebase_update_test.py
tests/lockfile_test.py
tests/metrics_test.py
tests/presubmit_canned_checks_test.py
tests/presubmit_unittest.py
tests/roll_dep_test.py
Change-Id: I8fed04b4ba81d54b8f45da612213aad27a9e1a2c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4842592
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
This is a reland of commit b5c7f4b46c
Original change's description:
> Drop py2 support in gerrit and git related files
>
> python3 is the only supported version of python in depot_tools.
>
> Bug: 1475402
> Change-Id: Ie4ee18d297081b3aa0206b8d7ce6461819bff0ca
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4809560
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
> Commit-Queue: Gavin Mak <gavinmak@google.com>
Bug: 1475402
Change-Id: I194180494071777b7b9dd91a5c8edabbbf5484c1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4811218
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
This reverts commit b5c7f4b46c.
Reason for revert: missing a replace for urlparse.urlparse
Original change's description:
> Drop py2 support in gerrit and git related files
>
> python3 is the only supported version of python in depot_tools.
>
> Bug: 1475402
> Change-Id: Ie4ee18d297081b3aa0206b8d7ce6461819bff0ca
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4809560
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
> Commit-Queue: Gavin Mak <gavinmak@google.com>
Bug: 1475402
Change-Id: Idd00fdfe0b3d62785da2789a7dfcc9fbc79b6385
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4811623
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
python3 is the only supported version of python in depot_tools.
Bug: 1475402
Change-Id: Ie4ee18d297081b3aa0206b8d7ce6461819bff0ca
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4809560
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
This reverts commit d9011c559b.
Reason for revert: Breaks ChromeOS staging builders: b/296139378
Original change's description:
> Set --git-dir for git commands that may be executed in bare gits.
>
> Bug:b/294415576
> Change-Id: I18ca8ebebf95e1c31e30aa1f5d62da3467df940f
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4778199
> Auto-Submit: Joanna Wang <jojwang@chromium.org>
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Joanna Wang <jojwang@chromium.org>
Bug: b/294415576
Change-Id: Ie16f16a405fbdea4d925e03a0cfd1ac0260bb2d8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4784102
Commit-Queue: Jack Neus <jackneus@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
This is a reland of d3affaa624
Original change's description:
> Use OS level locking in git_cache.py
>
> Without OS level locking it's possible to leave "lock" files on disk
> which will prevent next run to acquire those locks. This can easily
> happen if SIGKIL is issued.
>
> R=apolito@google.com, ehmaldonado@chromium.org
>
> Bug: 1049610
> Change-Id: Id87aa1376b9ea5ff0c2d14f3603636493ed1dd5b
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2189333
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Bug: 1049610
Change-Id: I58e65a10f7c779e0de1121ba7167c694996e390c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2211189
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
This reverts commit d3affaa624.
Reason for revert: no attribute ignore_lock
Original change's description:
> Use OS level locking in git_cache.py
>
> Without OS level locking it's possible to leave "lock" files on disk
> which will prevent next run to acquire those locks. This can easily
> happen if SIGKIL is issued.
>
> R=apolito@google.com, ehmaldonado@chromium.org
>
> Bug: 1049610
> Change-Id: Id87aa1376b9ea5ff0c2d14f3603636493ed1dd5b
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2189333
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
> Commit-Queue: Josip Sokcevic <sokcevic@google.com>
TBR=iannucci@chromium.org,ehmaldonado@chromium.org,apolito@google.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com
Change-Id: Iecc963e0a99d7f59f3f8801e529839346f9fbaf3
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1049610
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2211186
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Without OS level locking it's possible to leave "lock" files on disk
which will prevent next run to acquire those locks. This can easily
happen if SIGKIL is issued.
R=apolito@google.com, ehmaldonado@chromium.org
Bug: 1049610
Change-Id: Id87aa1376b9ea5ff0c2d14f3603636493ed1dd5b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2189333
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
If a local branch tracks removed remote tracking branch (e.g. foo) and
remote has a branch that starts with such name (e.g. foo/bar), git fetch
will fail. --prune flag removes any remote-tracking branches that no
longer exist.
R=apolito@google.com, ehmaldonado@chromium.org
Bug: 1079483
Change-Id: I9bc31bf961d52a86b6fa2342249971b99a003666
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2190341
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
This can happen if the cache repo was init'd, but has no pack files;
previously it would try to fetch from scratch.
Change-Id: I71689e30bdede392588c69e118e9297d86a134a3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2120281
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
There are 3 layers modified, starting from the bottom up:
1. git_cache.py populate
Now takes a --no-fetch-tags option. If specified, the cache will not
fetch updated tags from the server by passing --no-tags to git fetch.
This prevents the server from sending all the tag refs. In chromium
this prevents significant time bottlenecks dealing with 10k+ tags.
2. bot_update.py
bot_update has to deal with multiple git repos, it has the root repo
that is checked out through git-cache, as well as repos synched via
DEPS and gclient.
The script now takes a --no_fetch_tags option. If specified,
the git-cache populate call will include --no-fetch-tags. Otherwise, it
won't. So this controls (for chromium) if fetches to the src.git server
are passed --no-tags.
3. bot_update/api.py
This is the entry point for recipes and also has to deal with multiple
git repos. The behaviour at this point is not changed from the default.
A |no_fetch_tags| parameter is added to ensure_checkout() that defaults
to False.
This CL is a refactor with no intended behavior change.
The next step will be to change the chromium build repo to pass True
for ensure_checkout(no_fetch_tags) on chromium trybots.
This represents solution #2 in
https://docs.google.com/document/d/1hDIunJjjfpmr50y3YnZ4o3aq1NZF4gJa1TS0p7AIL64/edit#
Bug: 1019824
Change-Id: I935430603299daa9e301a95a5184c0ce486fd912
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1894352
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Erik Chen <erikchen@chromium.org>
Commit-Queue: danakj <danakj@chromium.org>
Auto-Submit: danakj <danakj@chromium.org>
We should only be maintaining one cache bundle per repo, but it turns
out that we've had two in the past due to GoB supporting two different
paths to the repo, and users were getting stale bundles as a result.
This CL fixes things so that we should only get a single bundle per
repo.
Bug: 935084
Change-Id: I0d6713280a2abbc20e35ff87e7be115870dd5140
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1566431
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
This CL makes a couple changes:
* The goal here is to be able to specify git cache entirely from the
environment variable $GIT_CACHE_PATH and not require special treatment from
bot_update. Eventually this will be specified at the swarming task level
instead of in the recipe (i.e. "cached git" will eventually be an
implementation detail of git on the bots and completely transparent to
all other software).
* Removal of the general --cache-dir option from gclient. This option was
error-prone; it doesn't actually make sense to configure this on
a per-invocation basis. The sole exception was `gclient config`, which
now allows this option to be set directly.
* Consolidation of GitWrapper.cache_dir and GitWrapper._GetMirror; previously
these two things could disagree, leading to weird intermediate states. Now
they're the same value.
R=agable@chromium.org, ehmaldonado@chromium.org, tandrii@chromium.org
Bug: 823434
Change-Id: I14adc7619b5fc10768ce32be2651c6215ba94aff
Reviewed-on: https://chromium-review.googlesource.com/1105473
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
With this change, 'git cache fetch' will automatically re-bootstrap
a repo from google storage if the local cache has too many pack
files. The behavior can be disabled with the --no-bootstrap flag.
BUG=
Review URL: https://codereview.chromium.org/1320383004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@296829 0039d316-1c4b-4281-b951-d872f2087c98
Jinli Wu
Josip Sokcevic
Mike Frysinger
Gavin Mak
Joanna Wang
Jack Neus
Anthony Polito
Josip Sokcevic
Edward Lesmes
danakj
Edward Lemur
Dirk Pranke
Robert Iannucci
szager@chromium.org