Adds 'gclient setvar' and 'gclient setdep' commands to edit variables
and dependencies in a DEPS file.
Bug: 760633
Change-Id: I6c0712cc079dbbbaee6541b7eda71f4b4813b77b
Reviewed-on: https://chromium-review.googlesource.com/950405
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
e.g. when specifying both '--revision src@<something>' and
'--revision <src_gerrit_repo>@<some_gerrit_ref>', gclient will sync to
'<some_gerrit_ref>' instead of '<something>'.
This is useful when specifying which gerrit change to patch, which
should take priority over other revision specifications.
Bug: chromium:643346
Change-Id: Ibc21ede355b56e4da966f38f144ce6f6f1743403
Reviewed-on: https://chromium-review.googlesource.com/949981
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Add a --output-json flag to gclient revinfo so is easier for other
programs to process its output.
Bug: None
Change-Id: Ic120e7a279f3ed537ead624ab9bfd1f141485487
Reviewed-on: https://chromium-review.googlesource.com/952206
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Add --path and --url options to show only the dependencies that match
one of the given paths or URLs.
Bug: None
Change-Id: I12c205545b7ce54b56abcd62f9bf1db229b4fd77
Reviewed-on: https://chromium-review.googlesource.com/951963
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
When hooks multiple cipd packages under same directory, gclient would
pop up error that same directory name appears multiple times in deps
section. This CL overrides verify_validity for CipdDependency to
support multiple packages hooked in same directory.
Bug:812386
Change-Id: Ia4f1fe0e3c8481c9b06c1d22b6e98d98e1e4c309
Reviewed-on: https://chromium-review.googlesource.com/920686
Commit-Queue: Shenghua Zhang <shenghuazhang@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
For now 'version' of packages only support string like 'version:27.0.5'.
It is helpful to support version to match vars value, e.g.
Var('tool_version'), especially useful when version numbers could be
modified via api. This change won't affect versions with string format.
Change-Id: I40b389b528f04712798b08f1455869cab2ec54c9
Reviewed-on: https://chromium-review.googlesource.com/935666
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Shenghua Zhang <shenghuazhang@chromium.org>
This CL adds target_cpu and target_cpu_only variables. The logic is:
* If target_cpu is defined in .gclient, use it. Otherwise, initialize it as an
empty list.
* If not target_cpu_only, add the host arch to the list too.
detect_host_arch.py is copied directly from the Chromium repo.
BUG=807986
R=dpranke
Change-Id: I27621cbc81ad6a844648525863b92ffdd3b1d03a
Reviewed-on: https://chromium-review.googlesource.com/902441
Commit-Queue: Thomas Anderson <thomasanderson@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Flattening currently ignores custom_vars, but it looks like that was
just an attempt to avoid including all the runtime-calculated values
(e.g. the host OS) in the flattened output, and not because there's any
real issue with supporting explicit overrides.
This is useful for changing default values in the flattened buildspecs
(e.g. to enable internal checkouts by default).
R=dpranke@google.com, iannucci@google.com
Bug: 570091,807318
Change-Id: I6d743f33cf7e3e202d5e68d21657f74f0e4c001b
Reviewed-on: https://chromium-review.googlesource.com/895927
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Michael Moss <mmoss@chromium.org>
This will allow us to show the progress when cloning a new repo when running
commands like gclient sync.
R=agable@chromium.org
Bug: 722686
Change-Id: I268cba343ea4c3c024292c9341d5009aa112b184
Reviewed-on: https://chromium-review.googlesource.com/890524
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
bot_update & gclient on bots instruct git cache locks
to be ignored, so there is no need to clean them up.
R=hinoka@chromium.org
Change-Id: I81c52cfcd44a42d13b641847439ac92053a7bc98
Reviewed-on: https://chromium-review.googlesource.com/791652
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
This will let progress show the name of the hook while it's running.
Change-Id: I14ef2a9b5fd2bc4d8d778f5402f78a5827ffbb08
Reviewed-on: https://chromium-review.googlesource.com/731469
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Commit-Queue: Daniel Cheng <dcheng@chromium.org>
When 'gclient recurse' is called from a unconfigured directory.
No functional change otherwise.
R=agable@chromium.org
Bug:774749
Change-Id: Ifb2b74c042ce31faf3e1287c5614739e58e9178c
Reviewed-on: https://chromium-review.googlesource.com/726419
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Marc-Antoine Ruel <maruel@chromium.org>
Previously, gclient would attempt to write an args file after
a dependency was checked out, but before any sub-dependencies
had been checked out. If the args file path pointed at something
inside a sub-dependency, this wouldn't work, because the directory
might not yet exist. This most obviously happened for buildspec
clobber builds.
The fix is to wait until after the sub-dependencies have been
checked out to write the file.
R=phajdan.jr@chromium.org, mmoss@chromium.org
BUG=773933
Change-Id: I0cf4564204f7dabd9f843dc7904db7050fcc0d23
Reviewed-on: https://chromium-review.googlesource.com/714644
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
These aren't in use, and the original problem they were
meant to solve has been solved at the gclient.py layer
using resource locking:
https://codereview.chromium.org/2049583003
Bug: 773008
Change-Id: I6609f39d7f15604e0bb3d742a41c4f9fec87a57a
Reviewed-on: https://chromium-review.googlesource.com/707728
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
Bug: 661382
Change-Id: Id50b2541132002452bc5d86bb013758e8be0f4b0
Reviewed-on: https://chromium-review.googlesource.com/697813
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
- provide structured output
- take deps_file fallback into account
Bug: 756474
Change-Id: Icb15eb9601b0aaf510300cf8992b067a25f6888a
Reviewed-on: https://chromium-review.googlesource.com/663258
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
This CL includes this as comments in flattened DEPS, as requested
on the bug. For automated processes using this, we should export
the data in machine-readable form outside of the DEPS file.
Bug: 756474, 570091
Change-Id: I78cd2105113f41d599e293e772e1f1ca42679f3c
Reviewed-on: https://chromium-review.googlesource.com/621726
Reviewed-by: Michael Moss <mmoss@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
We need to pass OS info to recursively called _flatten_dep.
Regular deps entries in OS-specific DEPS file recursed into
need to be marked as OS-specific in the flattened file.
Bug: 570091
Change-Id: If3055b84143d8a52d10d8753113893b5054b4d07
Reviewed-on: https://chromium-review.googlesource.com/621046
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
This fixes an issue where "--deps=all" syncs everything but ios.
R=dpranke@chromium.org, phajdan.jr@chromium.org
Change-Id: I413269c5026b2c04f0d34d9ca874057eb35e1076
Reviewed-on: https://chromium-review.googlesource.com/618802
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Michael Moss <mmoss@chromium.org>
Bug: 570091
Change-Id: I773b74b042233efa2a525f5f47e920468b7fea4a
Reviewed-on: https://chromium-review.googlesource.com/618930
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Consistently pin all dependencies - including deps_os and ones
with shortened shas.
We add --process-all-deps switch so that users can easily
tell gclient to check out all affected dependencies locally.
Bug: 570091
Change-Id: If68db98000c569ae35dd7d0a4b695eb80a589213
Reviewed-on: https://chromium-review.googlesource.com/617224
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
There are some usecases where it is simpler to pass path to the
cache by an environment variable or command-line argument
than to modify the gclient config file.
With this change:
* GCLIENT_CACHE_DIR environment variable overrides cache_dir variable
from .gclient file.
* --cache-dir argument moved from gclient config to all gclient
commands and overrides values from .gclient file and environment
variable.
R=dpranke@chromium.org
Bug:
Change-Id: Iae107f1d5f02a2eee1896d0d3108dc20a83d597f
Reviewed-on: https://chromium-review.googlesource.com/596089
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Commit-Queue: Aleksandr Derbenev <alex-ac@yandex-team.ru>
Otherwise it'd get stringified to "None" and make resulting DEPS invalid.
In theory we could properly serialize them to None,
but it's simpler to just skip them, since they do not affect resulting checkout.
Bug: 570091
Change-Id: Ica301e2d7e40ba4ed04527355d8f75a43456391c
Reviewed-on: https://chromium-review.googlesource.com/610150
Reviewed-by: Michael Moss <mmoss@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
This finally makes the code structure seem right as well: there's just
one method (_flatten_dep), with a simpler control flow.
Also added a regression test.
Bug: 570091
Change-Id: I22ac7a3af0429a7ffd874b4b1715c0f6c72e0006
Reviewed-on: https://chromium-review.googlesource.com/608241
Reviewed-by: Michael Moss <mmoss@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Nice side-effect of this change is simplifying the code.
Also added regression test coverage.
Bug: 570091
Change-Id: I470e9efc319632f997b02d210483988c17a7d3c8
Reviewed-on: https://chromium-review.googlesource.com/600369
Reviewed-by: Michael Moss <mmoss@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
- allow variables with same names as long as values are also the same
- allow recursedeps entries referencing deps_os entries
Bug: 570091
Change-Id: I4abbd255982368a0ee33e8f3de631f3d18adf6ab
Reviewed-on: https://chromium-review.googlesource.com/583649
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
This is part of preparation for gclient flatten, conditions,
and improving internal gclient workflows.
This also ensures variables have consistent values in the entire checkout,
unlike the current situation where different DEPS files can use different
values for the same variable.
We override instead of erroring out, since otherwise commonly used dependencies
such as buildtools wuld have to be rolled simultaneously in unrelated projects,
say v8 and native_client.
Another reason for overriding are possible future use cases such as is_chromium_build,
which could be set to false e.g. in v8, but chromium DEPS would override it to true.
Bug: 570091
Change-Id: If61322dd9a55e57729a4100d05ecb9c1e3ae8e38
Reviewed-on: https://chromium-review.googlesource.com/596310
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
This is an exact reland of https://chromium-review.googlesource.com/583617 .
One of the main use cases is making it clear which revision hashes
need to be changed together. The way it's usually done is one variable
referenced several times. With this CL, we preserve the references
from original DEPS, as opposed to evaluating them and losing some info.
This CL actually makes Var() emit a variable placeholder
instead of its value, and adds support for these placeholders
to gclient.
One of possible next steps might be to deprecate Var().
Bug: 570091, 748486
Change-Id: Id47e3771b7163149a4cd427b84f84ece52772f34
Reviewed-on: https://chromium-review.googlesource.com/586594
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: William Hesse <whesse@google.com>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
This reverts commit e79ddeaabf.
Reason for revert: https://bugs.chromium.org/p/chromium/issues/detail?id=748486
Original change's description:
> gclient flatten: preserve variable placeholders
>
> One of the main use cases is making it clear which revision hashes
> need to be changed together. The way it's usually done is one variable
> referenced several times. With this CL, we preserve the references
> from original DEPS, as opposed to evaluating them and losing some info.
>
> This CL actually makes Var() emit a variable placeholder
> instead of its value, and adds support for these placeholders
> to gclient.
>
> One of possible next steps might be to deprecate Var().
>
> Bug: 570091
> Change-Id: I9b13a691b5203cc284c33a59438720e31c9ebf7a
> Reviewed-on: https://chromium-review.googlesource.com/583617
> Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>
TBR=phajdan.jr@chromium.org,dpranke@chromium.org
Change-Id: If9c52ebfa78aba8041ce797ff842d09952d0e2ce
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 570091, 748486
Reviewed-on: https://chromium-review.googlesource.com/584907
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
One of the main use cases is making it clear which revision hashes
need to be changed together. The way it's usually done is one variable
referenced several times. With this CL, we preserve the references
from original DEPS, as opposed to evaluating them and losing some info.
This CL actually makes Var() emit a variable placeholder
instead of its value, and adds support for these placeholders
to gclient.
One of possible next steps might be to deprecate Var().
Bug: 570091
Change-Id: I9b13a691b5203cc284c33a59438720e31c9ebf7a
Reviewed-on: https://chromium-review.googlesource.com/583617
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Added a regression test. Simplified some logic - if we don't add os-specific
deps and hooks to |dependencies|, we don't need to keep separate original values.
Bug: 570091
Change-Id: I5bdd0b6a66df6b3a2b99d0ad9c6e54ee7114f09b
Reviewed-on: https://chromium-review.googlesource.com/581687
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
Instead of passing around 6+ out-parameters, keep state inside the class.
No intended behavior change.
Bug: 570091
Change-Id: Ia15c1db2170680f0ec087b61f51af49e12fd7579
Reviewed-on: https://chromium-review.googlesource.com/561700
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
It turns out it most likely won't be needed. Instead, auto-pinning
deps is useful mode.
This removal helps simplify and restructure the flatten code.
Bug: 570091
Change-Id: Ia70221c88a37f210cf1338a36d79168d2cfe96f6
Reviewed-on: https://chromium-review.googlesource.com/561717
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
The main goal is to provide a migration path, where we can add
conditional src-internal entry to src/DEPS, and have it still
work on checkouts where .gclient pulls src-internal, provided
the condition evaluates to False.
The migration path is then to remove the .gclient entry, and
at the same time flip the condition to True (e.g. by overriding
a variable).
Bug: 570091
Change-Id: I9b9850a644463ab0b1f368d65a5cd5f47cf7be97
Reviewed-on: https://chromium-review.googlesource.com/559150
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
The previous code did not distinguish between None and [] value for orig_deps.
This could be triggered by not having any deps, but some entries in deps_os,
in which case these deps_os entries would get added by mistake to deps.
Bug: 570091
Change-Id: Id5b41e37ea717d5d00404694ad5bfce8df9da934
Reviewed-on: https://chromium-review.googlesource.com/558349
Reviewed-by: Michael Moss <mmoss@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
R=dpranke@chromium.org, phajdan.jr@chromium.org
Bug: 570091
Change-Id: Ic2aa1a9fe18f3fe8d5aa6fa4c4e9269106b36092
Reviewed-on: https://chromium-review.googlesource.com/553719
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Di Mu <dimu@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Bug: 570091
Change-Id: Ia7f81a81d7df75004c5f8b7560dfd50a14f4cddd
Reviewed-on: https://chromium-review.googlesource.com/549355
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
Bug: 570091
Change-Id: Iae9dad68a75d751ceac6379baac588f32c59aa06
Reviewed-on: https://chromium-review.googlesource.com/548935
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
This is a reland of https://chromium-review.googlesource.com/c/541280/
with a fix for https://bugs.chromium.org/p/chromium/issues/detail?id=735418
(patchset 1 is original patch, patchset 2 has the fix).
Keep deps_os entries in dependencies, just with should_process set to False
for entries not applicable to target OS list.
This way gclient flatten has proper access to dependencies e.g. to evaluate
recursedeps referring to deps_os entries other than active OS.
Allow but ignore deps_os overriding a value with None, since that does not
fit the new model. There's no correctness harm in not checking out a repo.
Allow "overrides" setting given dependency to the same value. This seems
fairly common, especially for mac/ios and unix/android, even in chromium/src.
Bug: 570091, 735418
Change-Id: I6eba0e4be202212eb86cb959c18f2b2f0c1452b9
Reviewed-on: https://chromium-review.googlesource.com/543076
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This reverts commit 529d6a4e4a.
Reason for revert: broke developers and CI/Try checkouts.
Bug: 735418
Original change's description:
> gclient: include deps_os entries in dependencies
>
> Keep deps_os entries in dependencies, just with should_process set to False
> for entries not applicable to target OS list.
>
> This way gclient flatten has proper access to dependencies e.g. to evaluate
> recursedeps referring to deps_os entries other than active OS.
>
> Allow but ignore deps_os overriding a value with None, since that does not
> fit the new model. There's no correctness harm in not checking out a repo.
>
> Allow "overrides" setting given dependency to the same value. This seems
> fairly common, especially for mac/ios and unix/android, even in chromium/src.
>
> Bug: 570091
> Change-Id: I2037a1ecc5fd2da6b5f73061548b81fc79ba2e72
> Reviewed-on: https://chromium-review.googlesource.com/541280
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>
> Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
TBR=phajdan.jr@chromium.org,dpranke@chromium.org
Change-Id: Iaa0c39865908a5b25c15dda54ba61c0e76abcbea
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 570091
Reviewed-on: https://chromium-review.googlesource.com/543138
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Keep deps_os entries in dependencies, just with should_process set to False
for entries not applicable to target OS list.
This way gclient flatten has proper access to dependencies e.g. to evaluate
recursedeps referring to deps_os entries other than active OS.
Allow but ignore deps_os overriding a value with None, since that does not
fit the new model. There's no correctness harm in not checking out a repo.
Allow "overrides" setting given dependency to the same value. This seems
fairly common, especially for mac/ios and unix/android, even in chromium/src.
Bug: 570091
Change-Id: I2037a1ecc5fd2da6b5f73061548b81fc79ba2e72
Reviewed-on: https://chromium-review.googlesource.com/541280
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
This makes it possible to run hooks properly
in flattened DEPS.
Bug: 570091
Change-Id: If8175a57ebe8f607bd4ac83d4a26dcc4cc18165c
Reviewed-on: https://chromium-review.googlesource.com/535476
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
This will be useful e.g. to add cwd support for flatten.
No intended behavior change.
Bug: 570091
Change-Id: I014f97739676d55f6d5b37c10afd9221b1d0978d
Reviewed-on: https://chromium-review.googlesource.com/534193
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
It's unused, and not covered by tests.
See https://codereview.chromium.org/9232068/ where it was added ~5 years ago
for an abandoned project to convert to repo.
Bug: 570091
Change-Id: Ica59cd3b28f92e05607203218cbeb92a377ec99a
Reviewed-on: https://chromium-review.googlesource.com/534313
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
TBR=dpranke
Bug: 570091
Change-Id: I1b8cc8156c83c72b2fc599d90059bc7f2674ca17
Reviewed-on: https://chromium-review.googlesource.com/532995
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
This will also be useful for other values (deps_os, hooks_os)
we may want to store and keep constant.
Freeze code copied from luci/recipes-py @ 944125e6d1e8c831d09517bde658a38d8f81db37
Bug: 570091
Change-Id: I3365cf2b267c478316870bbb3fd41e9955aa4ddf
Reviewed-on: https://chromium-review.googlesource.com/531166
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This way we can get e.g. ordered dict as needed for conditions.
Only the new logic does it, not the regular python exec logic.
Bug: 570091
Change-Id: Ia5554e5b018085b3b9bd876b7f28a9f8e54a7984
Reviewed-on: https://chromium-review.googlesource.com/522564
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Pinpoint needs to request builds at specific revisions, e.g. clank@0123 + chromium/src@4567 + v8@89ab. It's not aware of the path of the repositories in the src tree, so it makes more sense to pass in the repository URLs instead.
Bug: https://github.com/catapult-project/catapult/issues/3394
Change-Id: Ib9fb9a7bf5d918c18f37c3b2b36418537512b235
Reviewed-on: https://chromium-review.googlesource.com/520569
Commit-Queue: David Tu <dtu@chromium.org>
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
This relands the safe portion of outage-causing https://chromium-review.googlesource.com/c/509451/
Landing this simple CL will allow pre-emptively adding escape hatch
to critical places (such as update_scripts) to make relanding validation safer.
Bug: 570091
Change-Id: I4d963bd9b140b7b6bca35d4d5e41768bc1a3d7e0
Reviewed-on: https://chromium-review.googlesource.com/512744
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Original CL: https://chromium-review.googlesource.com/509614
This feature seems to be unused, and removing it will
simplify handling hooks a little bit.
Bug: 661382
Change-Id: I7f02b34fc399c78bef76420df19a93105ab098ac
Reviewed-on: https://chromium-review.googlesource.com/512644
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
This reverts commit 3e6d7c1cbc.
Reason for revert: maybe caused outage.
Original change's description:
> gclient: remove support for $matching_files in hooks
>
> This feature seems to be unused, and removing it will
> simplify handling hooks a little bit.
>
> Bug: 661382
> Change-Id: I89f28dedb7f59cd475b176cfb1f023094520d6b7
> Reviewed-on: https://chromium-review.googlesource.com/509614
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>
> Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
>
TBR=phajdan.jr@chromium.org,dpranke@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
Bug: 661382
Change-Id: I47fe26e7381682b5b428a3775bf27a551c57d5e6
Reviewed-on: https://chromium-review.googlesource.com/512344
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
This feature appears unused, and removing it will simplify the codebase.
Bug: 661382
Change-Id: I545befb2c592eea53c54552018ce2d3dda7670f5
Reviewed-on: https://chromium-review.googlesource.com/509693
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
This feature seems to be unused, and removing it will
simplify handling hooks a little bit.
Bug: 661382
Change-Id: I89f28dedb7f59cd475b176cfb1f023094520d6b7
Reviewed-on: https://chromium-review.googlesource.com/509614
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
This is based on https://codereview.chromium.org/2474543002/
Bug: 661382
Change-Id: I191ec16e0ce69a782979ae7d59b108747429ab78
Reviewed-on: https://chromium-review.googlesource.com/505067
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
This adds support for 'hooks_os' in .gclient which runs the given hooks
only when the associated os is specifed in target_os.
Bug: 706592
Change-Id: If70e51e0e784f8a8c6e45b33f59605b883a16f6e
Reviewed-on: https://chromium-review.googlesource.com/503534
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Currently if you set `pre_deps_hooks` in your gclient DEPS file,
they will be executed regardless of whether your repo was supposed
to be recursed into (via recursionlimit or recursedeps) or not.
This change fixes that so that we only run the hooks that were
explicitly configured to be run.
R=kjellander@chromium.org, agable@chromium.org
BUG=696495
Change-Id: Ic0ef641903cf3f56d7c2e119d8b0db862258d995
Reviewed-on: https://chromium-review.googlesource.com/447318
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Henrik Kjellander <kjellander@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
This affects a bunch of files, but only changes comments,
and shouldn't make any difference to behavior.
The purpose is to slightly improve readability of pylint
disable comments.
Change-Id: Ic6cd0f8de792b31d91c6125f6da2616450b30f11
Reviewed-on: https://chromium-review.googlesource.com/420412
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Currently, if a DEPS file sets use_relative_paths, but *also* sets
recursedeps, then the recursed-upon DEPS files still get checked
out relative to the .gclient root. This change makes it so that
recursed-upon DEPS files check their dependencies out relative to
where their parent wants them to be, if that parent sets
use_relative_paths=True.
R=maruel@chromium.org
Review-Url: https://codereview.chromium.org/2324513003
This is a workaround for the fact that 'gclient revert' calls
'gclient update' with the same `options` object, despite not
actually having the same set of command line options.
R=iannucci@chromium.org, kevcheng@chromium.org
BUG=631220
Review-Url: https://codereview.chromium.org/2184493002
There are entries in the DEPS file where two folders uses the same
git URL (ie. freetype2). This doesn't work well with git caches because
each task will run on it's own and might try to clobber on top of each other.
This adds another field in a WorkItem which is a list of resources. When the
work queue is flushed, it has to make sure that none of a newly added workitem
has any resource conflicts.
BUG=618124
Review-Url: https://codereview.chromium.org/2049583003
BUG=591814
TEST=gclient sync prints the correct report locally
when config is wrong.
Review-Url: https://codereview.chromium.org/2018803002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@300671 0039d316-1c4b-4281-b951-d872f2087c98
This will allow ANGLE to have a recursible deps-file in their repo instead of
relying on chromium's DEPS file to specify an accurate dependency for itself.
Previously: https://chromiumcodereview.appspot.com/1919103003/
R=agable@chromium.org, dpranke@chromium.org, jmadill@chromium.org
BUG=
Review-Url: https://codereview.chromium.org/1948853002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@300446 0039d316-1c4b-4281-b951-d872f2087c98
This will allow ANGLE to have a recursible deps-file in their repo instead of
relying on chromium's DEPS file to specify an accurate dependency for itself.
R=agable@chromium.org, jmadill@chromium.org
BUG=
Review-Url: https://codereview.chromium.org/1919103003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@300293 0039d316-1c4b-4281-b951-d872f2087c98
CMDupdate and CMDsync have different option parsers, which means they pass
different options objects to the same underlying code!
R=agable@chromium.org, dnj@chromium.org, smut@chromium.org
BUG=601564
Review URL: https://codereview.chromium.org/1869593004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@299777 0039d316-1c4b-4281-b951-d872f2087c98
This will clean up index.lock and refs/heads/master.lock-style locks when
invoked as part of the update_scripts step on the bots.
R=dpranke@chromium.org, estaab@chromium.org, pgervais@chromium.org
BUG=421769
Review URL: https://codereview.chromium.org/1862943002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@299737 0039d316-1c4b-4281-b951-d872f2087c98
* `git bash` helper now will either:
* launch an interactive bash prompt in the MinGW Terminal Emulator
* launch a non-interactive bash shell if arguments are provided
* `git bash` now includes python, python/Scripts and svn binaries in
the PATH.
* depot_tools' colorization now works in MinGW Terminal Emulator
* batch scripts are now regenerated if any of them is missing.
R=agable@chromium.org, dnj@chromium.org, mmoss@chromium.org
BUG=598956
Review URL: https://codereview.chromium.org/1847783002 .
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@299616 0039d316-1c4b-4281-b951-d872f2087c98
gclient-sync + git-cache are racy because if two different DEPS files
dependent on the same repo URL, even if depsed to different checkout
dirs, they use same git cache dir. If both checkout dirs are synced at
the same time, one of them cannot acquire a lock.
This is a cheap solution to change --lock_timeout option default value
from 0 to 5m.
R=hinoka@chromium.org
BUG=593468
Review URL: https://codereview.chromium.org/1785083005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@299386 0039d316-1c4b-4281-b951-d872f2087c98
I guess I'm the only developer using git-cache, which is sad.
Hopefully these fixes will make it easier to adapt this to developer
usage some time in the FUTURE.
BUG=583420
TEST=Works for me
R=agable@chromium.org,tandrii@chromium.org
Review URL: https://codereview.chromium.org/1650993005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@298531 0039d316-1c4b-4281-b951-d872f2087c98
Reason for revert:
breaks git-bash on windows
Original issue's description:
> git_cl/gclient: use python2
>
> Newer distros are defaulting /usr/bin/python to python3, so use python2
> explicitly so we continue working.
>
> Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=297535
TBR=stip@chromium.org,sergeyberezin@chromium.org,vapier@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
Review URL: https://codereview.chromium.org/1442583004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@297544 0039d316-1c4b-4281-b951-d872f2087c98
Newer distros are defaulting /usr/bin/python to python3, so use python2
explicitly so we continue working.
Review URL: https://codereview.chromium.org/1437773002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@297535 0039d316-1c4b-4281-b951-d872f2087c98
The newer form works with python2 and python3.
Review URL: https://codereview.chromium.org/1436763002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@297517 0039d316-1c4b-4281-b951-d872f2087c98
This makes the code work both in python2 and python3.
Review URL: https://codereview.chromium.org/1433153003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@297516 0039d316-1c4b-4281-b951-d872f2087c98
This helps prevent behavior such as setting '--name .' so that the .gclient
file contains '.' as the name of the solution, and the .gclient file ends
up a sibling of the .git file, rather than a sibling of the git checkout's
containing directory.
R=iannucci@chromium.org, thestig@chromium.org
Review URL: https://codereview.chromium.org/1406053003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@297343 0039d316-1c4b-4281-b951-d872f2087c98
This cl ensures that gclient enforces sane revision overrides and
onfiguration when running a revert, just like it does on a normal sync/update.
R=iannucci@chromium.org
BUG=420919
Review URL: https://codereview.chromium.org/1160113003.
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295571 0039d316-1c4b-4281-b951-d872f2087c98
Handle KeyboardInterrupt gracefully rather the printing a
backtrace. Most users of these tools don't expect a
backtrace when then hit Ctrl-C.
Also, fix a few other inconsistencies found in the python
startup code of these different scripts:
- always call main function 'main' (rather than 'Main')
- always return 0 from main function
- if main takes args never include argv[0]
Review URL: https://codereview.chromium.org/955993006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294250 0039d316-1c4b-4281-b951-d872f2087c98
This adds the '--auto_rebase' sync option, enabling parallel updates to
automatically rebase local Git branches during sync.
BUG=None
TEST=local
Review URL: https://codereview.chromium.org/930693002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294151 0039d316-1c4b-4281-b951-d872f2087c98
Reland crrev.com/743083002, which was reverted in crrev.com/796053002
due to some test flakiness, probably related with an old version of Git on
the bots. Relanding now that the infra has been updated to Trusty (plus
adding some de-flake precautions).
Original CL Description:
Make gclient ready for the Blink (DEPS to main project) transition
This CL makes gclient understand correctly whether a git project is
being moved from DEPS to an upper project and vice-versa.
The driving use case for this is the upcoming Blink merge, where
third_party/Webkit will be removed from DEPS (and .gitignore) and will
become part of the main project.
At present state, gclient leaves the .git folder around when a project
is removed from DEPS, and that causes many problems.
Furthermore this CL solves the performance problem of bisecting across
the merge point. The subproject's (Blink) .git/ folder is moved to a
backup location (in the main checkout root) and is restored when moving
backwards, avoiding a re-fetch when bisecting across the merge point.
BUG=431469
Review URL: https://codereview.chromium.org/910913003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@294082 0039d316-1c4b-4281-b951-d872f2087c98
Reason for revert:
The test intoruced is flake (see crrev.com/753543006/).
Need to look into that with more patience as it reproduces only on precise.
Original issue's description:
> Make gclient ready for the Blink (DEPS to main project) transition
>
> This CL makes gclient understand correctly whether a git project is
> being moved from DEPS to an upper project and vice-versa.
> The driving use case for this is the upcoming Blink merge, where
> third_party/Webkit will be removed from DEPS (and .gitignore) and will
> become part of the main project.
>
> At present state, gclient leaves the .git folder around when a project
> is removed from DEPS, and that causes many problems.
>
> Furthermore this CL solves the performance problem of bisecting across
> the merge point. The subproject's (Blink) .git/ folder is moved to a
> backup location (in the main checkout root) and is restored when moving
> backwards, avoiding a re-fetch when bisecting across the merge point.
>
> BUG=431469
>
> Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=293329
TBR=iannucci@chromium.org,maruel@chromium.org,jochen@chromium.org
NOTREECHECKS=true
NOTRY=true
BUG=431469
Review URL: https://codereview.chromium.org/796053002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293352 0039d316-1c4b-4281-b951-d872f2087c98
This CL makes gclient understand correctly whether a git project is
being moved from DEPS to an upper project and vice-versa.
The driving use case for this is the upcoming Blink merge, where
third_party/Webkit will be removed from DEPS (and .gitignore) and will
become part of the main project.
At present state, gclient leaves the .git folder around when a project
is removed from DEPS, and that causes many problems.
Furthermore this CL solves the performance problem of bisecting across
the merge point. The subproject's (Blink) .git/ folder is moved to a
backup location (in the main checkout root) and is restored when moving
backwards, avoiding a re-fetch when bisecting across the merge point.
BUG=431469
Review URL: https://codereview.chromium.org/743083002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@293329 0039d316-1c4b-4281-b951-d872f2087c98
This is breaking the ability to specify an alternate DEPS file, and do
things like sync release tags where DEPS is still from svn and we need
.DEPS.git to do a git checkout.
R=iannucci@chromium.org, szager@chromium.org
BUG=407049
Review URL: https://codereview.chromium.org/511533003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@291665 0039d316-1c4b-4281-b951-d872f2087c98
This is primarily a hack to enable the chromeos ebuild.
BUG=367027
TBR=iannucci@chromium.org,mmoss@chromium.org
Review URL: https://codereview.chromium.org/490233003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@291075 0039d316-1c4b-4281-b951-d872f2087c98
Original CL: https://codereview.chromium.org/440263002/
Many people* have complained on chromium-dev about the long times
required to perform a full fetch over a DSL. This seems to be mostly
due to the huge size of chromium's history (~9 GB). On the other side,
not everybody is interested in downloading the full git history of
the projects. The size of git packs required to fetch a working HEAD
is one order of magnitude smaller (1.5 GB).
This change makes it possible to perform a shallow fetch (in a way
which is consistent with DEPS, leveraging git templates on clone),
reducing fetch times by 80% for those not interested in the history.
* See:
[chromium-dev] "fetch chromium" keeps hanging/getting stuck on Windows 7
[chromium-dev] Initial checkout with git taking long
[chromium-dev] Trying to get latest source code fails when fetching
[chromium-dev] Gclient sync takes too long
BUG=228996
TBR=iannucci@chromium.org,szager@chromium.org,wtc@chromium.org
Review URL: https://codereview.chromium.org/440273002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@287793 0039d316-1c4b-4281-b951-d872f2087c98
Reason for revert:
Broke the WebRTC waterfall:
http://build.chromium.org/p/tryserver.webrtc/builders/win/builds/3958/steps/gclient%20revert/logs/stdio
Original issue's description:
> Add --no-history option to fetch and gclient for shallow clones.
>
> Many people* have complained on chromium-dev about the long times
> required to perform a full fetch over a DSL. This seems to be mostly
> due to the huge size of chromium's history (~9 GB). On the other side,
> not everybody is interested in downloading the full git history of
> the projects. The size of git packs required to fetch a working HEAD
> is one order of magnitude smaller (1.5 GB).
> This change makes it possible to perform a shallow fetch (in a way
> which is consistent with DEPS, leveraging git templates on clone),
> reducing fetch times by 80% for those not interested in the history.
>
> * See:
> [chromium-dev] "fetch chromium" keeps hanging/getting stuck on Windows 7
> [chromium-dev] Initial checkout with git taking long
> [chromium-dev] Trying to get latest source code fails when fetching
> [chromium-dev] Gclient sync takes too long
>
> BUG=228996
>
> Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=287606
TBR=iannucci@chromium.org,szager@chromium.org,wtc@chromium.org
NOTREECHECKS=true
NOTRY=true
BUG=228996
Review URL: https://codereview.chromium.org/440263002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@287637 0039d316-1c4b-4281-b951-d872f2087c98
Many people* have complained on chromium-dev about the long times
required to perform a full fetch over a DSL. This seems to be mostly
due to the huge size of chromium's history (~9 GB). On the other side,
not everybody is interested in downloading the full git history of
the projects. The size of git packs required to fetch a working HEAD
is one order of magnitude smaller (1.5 GB).
This change makes it possible to perform a shallow fetch (in a way
which is consistent with DEPS, leveraging git templates on clone),
reducing fetch times by 80% for those not interested in the history.
* See:
[chromium-dev] "fetch chromium" keeps hanging/getting stuck on Windows 7
[chromium-dev] Initial checkout with git taking long
[chromium-dev] Trying to get latest source code fails when fetching
[chromium-dev] Gclient sync takes too long
BUG=228996
Review URL: https://codereview.chromium.org/437903002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@287606 0039d316-1c4b-4281-b951-d872f2087c98
Also cast recursedeps to a set. We'll continue
to support using a list literal form in DEPS for
now.
R=iannucci@chromium.org,agable@chromium.org
BUG=393000
Review URL: https://codereview.chromium.org/385123008
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@283474 0039d316-1c4b-4281-b951-d872f2087c98
Created the 'GitRefish' class to centralize 'git' refish parsing and consistent
usage by 'gclient' 'git' code.
BUG=373504
TEST=localtest
R=agable@chromium.org, iannucci@chromium.org
Review URL: https://codereview.chromium.org/328843005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@281553 0039d316-1c4b-4281-b951-d872f2087c98
Now that recurselist is no longer a list, it doesn't make
sense to call it recurselist. recurseset is available, but
that's not as easy to read/say compared to recurselist.
Call this recursedeps, instead.
R=iannucci@chromium.org
BUG=390246
Review URL: https://codereview.chromium.org/363103002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@281107 0039d316-1c4b-4281-b951-d872f2087c98
It's possible to tell gclient to use a different
"deps" file from the default DEPS through the "deps_file"
variable in the .gclient file.
If this file is missing, fallback to DEPS (the
default).
BUG=390700
Review URL: https://codereview.chromium.org/368713002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@280921 0039d316-1c4b-4281-b951-d872f2087c98
Previously, recursion overrides were only available by setting
a numeric 'depth' value in a DEPS file. This meant that it
was not possible to control recursion per-dependency entry.
This change adds a recurselist variable with a list structure.
If a named dependency is present in the list, then gclient will
recurse into that dependency's DEPS.
As part of this change, I move the recursion controls off of
DependencySetting and onto Dependency. The new setup of being
based on Dependency allows access to the dependency's name.
The controls are only called from Dependency instances. They
have always needed access to self.parent (in the Dependency
context), so this should be more correct than the previous setup.
BUG=390246
Review URL: https://codereview.chromium.org/331373009
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@280690 0039d316-1c4b-4281-b951-d872f2087c98
Currently, only top-level solutions may be pinned via --revision. With this, we can:
gclient sync --revision src/third_party/WebKit@deadbeef
BUG=
Review URL: https://codereview.chromium.org/290443003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@271571 0039d316-1c4b-4281-b951-d872f2087c98
This occurs for svn checkouts when the DEPS change from:
third_party/skia/gyp
third_party/skia/include
third_party/skia/src
to:
third_party/skia
First, gclient deletes/moves third_party/skia to make room for the new checkout.
When finished, it notices that the three subdirectories under skia are now
"unversioned" according to .gclient_entries, even though they are part of Skia
repo. On the bots, this causes them to be deleted.
There is a check to determine whether the thought-to-be unversioned directories
are actually subdirectories of another checkout, but it doesn't work in the svn
case because the check assumes that, if they are subdirectories of another
checkout, it must be an svn checkout. scm.GetCheckoutRoot() returns None
because it can't find an svn checkout containing those subdirectories. None is
not in gclient_entries, so the directories get deleted.
BUG=
Review URL: https://codereview.chromium.org/275103009
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@270133 0039d316-1c4b-4281-b951-d872f2087c98
So that bot_update can use them to figure out which repo correspons to where.
BUG=370503
Review URL: https://codereview.chromium.org/268373003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@268875 0039d316-1c4b-4281-b951-d872f2087c98
Evidence indicates that running non-builtin git commands is very
slow in msysgit, slow enough to dominate the running time of
gclient sync. With this change, gclient never shells out to
git-cache; it import the lib directly instead.
R=agable@chromium.org,hinoka@chromium.org
BUG=
Review URL: https://codereview.chromium.org/229653002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@262759 0039d316-1c4b-4281-b951-d872f2087c98
Features:
- Non-verbose output is now limited to a one-line progress
indicator.
- Verbose output is now collated per subprocess. As soon as a
subprocess finishes, its full output is dumped to terminal.
- Verbose output is prefixed with timestamps representing elapsed
time since the beginning of the gclient invocation.
- git progress indicators ("Receiving objects", etc.) are limited to
one line every 10 seconds.
- In both verbose and non-verbose mode, if a failure occurs, the
full output of the failed update operation is dumped to terminal
just before exit.
- In the event that updates are progressing, but slowly,
"Still working" messages will be printed periodically, to pacify
users and buildbots.
BUG=
R=hinoka@google.com
Review URL: https://codereview.chromium.org/227163002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@262500 0039d316-1c4b-4281-b951-d872f2087c98
Instead of having custom logic for dealing with cache directories, use
git_cache.py to populate caches.
Also fixes a bug in git_cache.py where it was looking for lockfiles in cwd rather than the cache dir.
Other changes:
* _Run now returns output.
* Always print to stdout in CheckCallAndFilterOutput, even if it gets a carriage return. This is done because git progress report are carriage returns and not newlines and we don't want everything on the same line and not strip out the CRs.
* Removed members changed tests, its not very useful to know a new import is added.
BUG=339171
Review URL: https://codereview.chromium.org/180243006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@254248 0039d316-1c4b-4281-b951-d872f2087c98
It's not true (anymore?) that the number of SCM jobs for gclient sync equals the
number of cores, thus the new text is less specific.
BUG=none
Review URL: https://codereview.chromium.org/184733007
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@254242 0039d316-1c4b-4281-b951-d872f2087c98
Given that we search for the gclient root in ancestor directories based on the file name, specifying a file with separators doesn't make a lot of sense.
BUG=336946
R=iannucci@chromium.org
Review URL: https://codereview.chromium.org/150633004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@249077 0039d316-1c4b-4281-b951-d872f2087c98
Reason for revert:
This breaks `gclient sync` for me. Before this patch:
________ found .git directory; skipping src
After this patch:
Error: 1> Can't update/checkout /Volumes/MacintoshHD2/src/chrome-git/src if an unversioned directory is present. Delete the directory and try again.
Original issue's description:
> If the destination directory doesn't contain the desired repo, delete it
>
> Basic functionality - this may not be in quite the right place.
>
> Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=245404
TBR=iannucci@chromium.org,ilevy@chromium.org,cmp@chromium.org,maruel@chromium.org,morrita@chromium.org,borenet@google.com
NOTREECHECKS=true
NOTRY=true
Review URL: https://codereview.chromium.org/141633005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@245530 0039d316-1c4b-4281-b951-d872f2087c98
Basic functionality - this may not be in quite the right place.
Review URL: https://codereview.chromium.org/61623008
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@245404 0039d316-1c4b-4281-b951-d872f2087c98
It's possible in gclient to list multiple operating systems on the
command line and while handling them all perfectly might not be possible
it's possible to make it better than today.
This patch makes sure None never overrides any previous value and it
adds some output to indicate what kind of dangerous choices it made.
BUG=248168
Review URL: https://codereview.chromium.org/23875029
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@240892 0039d316-1c4b-4281-b951-d872f2087c98
These are run for a given dependency after it has been synced but before its
DEPS have been synced. This will help to switch Chromium to depend on Skia's
git repository (skia:1638).
Review URL: https://codereview.chromium.org/25322002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@228651 0039d316-1c4b-4281-b951-d872f2087c98
This is in order to support 'features' like got_revision for build systems
unwilling to scrape the human-readable output of gclient in order to extract
basic information :)
R=agable@chromium.org, maruel@chromium.org
BUG=265618
Review URL: https://chromiumcodereview.appspot.com/23753008
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@225046 0039d316-1c4b-4281-b951-d872f2087c98
Update subcommand to support examples, always disable format_description and
format_epilog and add colors when enabled.
R=iannucci@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/22824018
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@218180 0039d316-1c4b-4281-b951-d872f2087c98
Convert monkey patching to class inheritance for OptionParser usage in:
commit_queue.py, gclient.py and trychange.py.
Monkey patching confuses the hell out of pylint.
R=jochen@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/19552004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@213423 0039d316-1c4b-4281-b951-d872f2087c98
We can use this to evaluate the usefulness of making hooks run in parallel.
Review URL: https://chromiumcodereview.appspot.com/18851005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@211446 0039d316-1c4b-4281-b951-d872f2087c98
Sometimes we wish to pull in a complicated dependency but want to
suppress or replace one or more of the hooks rules. Say for example
we want to use a different way of generating the projects, or
have a different set of landmine expectations.
Here we add a custom_hooks section mirroring custom_deps to allow us to
override sections we have identified in the DEPS file. To do so,
we add an optional name to the elements of the hooks list, and overwrite
those whose name matches.
Conventions between included DEPS and the .gclient as to the meanings of the
name are equivalent to the meaning of the customized deps, and so do not
benefit from further structure or definition.
BUG=None
TEST=local unit test
Review URL: https://chromiumcodereview.appspot.com/17742004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@210573 0039d316-1c4b-4281-b951-d872f2087c98
This enables significant time savings, especially since file_list only exists to
enable file-specific hooks (which, AFAIK, nothing actually uses). On a z620
(linux) using the cached git repos, a first-time `gclient sync --nohooks` takes:
* (with) 131.06s user 14.10s system 117% cpu 2:03.89 total
* (without) 482.13s user 189.35s system 144% cpu 7:45.63 total
This change makes nohooks cause file_list to be None if we don't need to
accumulate it, and updates GitWrapper and SvnWrapper appropriately.
R=szager@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/18541006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@210026 0039d316-1c4b-4281-b951-d872f2087c98
Instead of cloning straight into place, clones are made to a global cache dir,
and then local (using --shared) clones are made from the cache to the final
resting place. This means the 'final' clones are full repos with no shenanigans,
meaning that branches, commits, etc. all work, which should allow the rest of
the gclient ecosystem to work without change as well.
The primary benefit is, of course, reduced network IO, and a much lower cost for
'clobber' operations (assuming we don't clobber the cache). It also means that
a given bot can have a greater number of checkouts, since the entire git history
will only be stored once per machine, instead of once per checkout.
R=dpranke@chromium.org, szager@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/18328003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@210024 0039d316-1c4b-4281-b951-d872f2087c98
This will be used on the bots when calling apply_patch, which commits to git
repos.
R=dpranke@chromium.org,szager@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/18006002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@209040 0039d316-1c4b-4281-b951-d872f2087c98
This will speed up main waterfall sync. We can
remove this hack tomorrow if svn load is still OK.
TBR=cmp
Review URL: https://chromiumcodereview.appspot.com/17468009
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@207277 0039d316-1c4b-4281-b951-d872f2087c98
SVN traps SIGINT and attempts to clean itself up, but this results in hangs
waiting for TCP. This patch does two things: daemonizes worker threads so they
are culled when the main thread dies (is ctrl-C'd) and keeps track of spawned
subprocesses to kill any remaining ones when the main program is ctrl-C'd.
A user ctrl-C'ing gclient has to manually terminate hung SVN processes, so this
introduces no extra data loss or hazard. stracing a hung SVN process shows that
it is indeed hanging on TCP reads after receiving a SIGINT, implying there is an
underlying but in the SVN binary.
Review URL: https://chromiumcodereview.appspot.com/14759006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@198205 0039d316-1c4b-4281-b951-d872f2087c98
The 'RemoveDirectory()' function in gclient_utils is deprecated and
rmtree() should be used instead for consistency.
This patch modifies all clients in depot_tools to use rmtree() instead
and removes the RemoveDirectory function.
+ The SVNWrapperTestCase.testRevertNoDotSvn() mocking
expectation has been slightly changed. This was required
because the test invokes code that used to call
gclient_utils.RemoveDirectory() directly, while only
gclient_utils.rmtree() was mocked.
BUG=NONE
R=maruel@chromium.org, ilevy@chromium.org
TEST=manually run gclient_utils_test / gclient_smoketest / scm_unittest / gclient_scm_test
Review URL: https://chromiumcodereview.appspot.com/14134010
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@196133 0039d316-1c4b-4281-b951-d872f2087c98
Here's some background why we need this:
We discovered that google code defines the timestamp of a revision to be the time when a commit was started
rather than when it was finished (apache subversion takes the timestamp when the commit transaction is finished).
This can result in a situation where revision R(i-1) has a higher timestamp than Ri.
See bug: https://code.google.com/p/support/issues/detail?id=30419
When using 'gclient --transitive' we effectively do date-based checkouts.
If a parent has a dependency (without a ...@revision) and that dependency lives in the same repository as the parent does
we'd like to checkout the exact same revision as the parent (if we do a date-based checkout as we do now the google code
bug can result in a situation where we don't get the same revision).
Review URL: https://chromiumcodereview.appspot.com/13814012
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@194852 0039d316-1c4b-4281-b951-d872f2087c98
Some SVN-only trybots get errors when running gclient revert with this
change. I assume it's because the flag is not present for the gclient
revert command, and that the code checking if options.with_branch_heads
is set is common to both revert and sync.
BUG=none
TEST=reverted locally and gclient revert worked again.
Review URL: https://chromiumcodereview.appspot.com/12616006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@186967 0039d316-1c4b-4281-b951-d872f2087c98
The "branch-heads" are refspecs created from upstream svn branches. This flag
is mostly needed for official builders, and adds about 1/2GB of data to the
checkout, so not currently enabling by default.
Review URL: https://codereview.chromium.org/12546017
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@186598 0039d316-1c4b-4281-b951-d872f2087c98
Support 'git grep <match> <branch>' type searches in gclient grep.
These searches had a slightly different output so needed more
tweeking to the regex. I cleaned it up a bit too.
R=maruel@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/11316213
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@170039 0039d316-1c4b-4281-b951-d872f2087c98
- This will be used by navabi to have his gclient recurse
command print the current repo.
R=maruel@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/11308115
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@169812 0039d316-1c4b-4281-b951-d872f2087c98
This allows specificying that a target OS list should override the base
list instead of adding to it. This is useful for iOS builds, where the
desired behavior is to pull only the iOS deps, not the union of the Mac
and iOS deps.
Review URL: https://chromiumcodereview.appspot.com/11363036
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@165671 0039d316-1c4b-4281-b951-d872f2087c98
This will let DEPS files specify a higher local bound for recursion.
Note there is an edge case where a dependency can be initially pulled
in with a lower recursion level and then gets ignored subsequently
when a lower level DEPS overrides it's inclusion level and then
includes the same dependency. I do not solve this case, as
I am intending to use this syntax for top level deps files.
BUG=155780
Review URL: https://chromiumcodereview.appspot.com/11146032
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@162464 0039d316-1c4b-4281-b951-d872f2087c98
- On high core, fast machines, jobs=8 is does not offer good
parallelization. Switch to use number of local cpus.
R=maruel@chromium.org
BUG=
Review URL: https://chromiumcodereview.appspot.com/11140019
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@162072 0039d316-1c4b-4281-b951-d872f2087c98
Edward Lesmes
Shenghua Zhang
John Budorick
Tom Anderson
Michael Moss
Benjamin Pastene
Andrii Shyshkalov
Nodir Turakulov
David Benjamin
Daniel Cheng
Marc-Antoine Ruel
Dirk Pranke
Michael Achenbach
Vadim Shtayura
Ryan Tseng
Paweł Hajdan, Jr
Jaideep Bajwa
Aleksandr Derbenev
Dave Tu
Scott Graham
Aaron Gable
Andrey Kosyakov
Quinten Yearsley
sdefresne
lpique
qyearsley
smut
levarum@chromium.org
iannucci@chromium.org
martiniss@chromium.org
rwalker@codeaurora.org
nodir@chromium.org
sergiyb@chromium.org
thakis@chromium.org
szager@chromium.org
vapier@chromium.org
stip@chromium.org
e.hakkinen@samsung.com
sbc@chromium.org
dnj@chromium.org
primiano@chromium.org
mmoss@chromium.org
iannucci@chromium.org
szager@chromium.org
dyen@chromium.org
primiano@chromium.org
cmp@chromium.org
smut@google.com
dnj@chromium.org
hinoka@chromium.org
borenet@google.com
scr@chromium.org
rnk@chromium.org
hinoka@google.com
tnagel@chromium.org
mmoss@chromium.org
rdsmith@chromium.org
thakis@chromium.org
bratell@opera.com
scottmg@chromium.org
maruel@chromium.org
xusydoc@chromium.org
ilevy@chromium.org
petermayo@chromium.org
digit@chromium.org
kustermann@google.com
jochen@chromium.org
kjellander@chromium.org
bcwhite@chromium.org
sivachandra@chromium.org
stuartmorgan@chromium.org