A new-line token is being added on Python 3, so add it explicitly
so that it also appears on Python 2.
Bug: 984182
Change-Id: I3bfb6ca83d94b6a3452b938f35f5a3174fe85903
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1764465
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
A previous CL replaced "basestring" with "str", because basestring
does not exist in Python 3. However, this broke Python 2's ability
to interoperate with unicode strings. This CL introduces a workaround
(defining basestring to be equivalent to string, if it doesn't exist
already), and restores the references to basestring. This workaround
can be fixed when we're 100% on Python 3.
It also undoes some unnecessary and harder-to-read formatting changes.
Bug: 942522
Change-Id: I4a31ee46dc048134c2e4832b6c44ea00ce341899
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1572441
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
This enables gclient sync and gclient runhooks to run, barring hook script failures.
git cl upload also now works.
The scripts still work with Python 2.
There are no intended behaviour changes.
Bug: 942522
Change-Id: I2ac587b5f803ba7f5bb5e412337ce049f4b1a741
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1524583
Commit-Queue: Raul Tambre <raul@tambre.ee>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Make it possible to refer to built-in variables without having to declare
then in DEPS files.
Bug: None
Change-Id: I5403963052463befc074f29750de56cce13927ce
Reviewed-on: https://chromium-review.googlesource.com/c/1312234
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
When a recursive dependency has use_relative_paths it also makes sense
to have the hooks working directory by the dependency's directory.
Otherwise if a hook uses one of the relative dependencies it is impossible
to know which path prefix to use.
However we cannot change the behavior of hooks with use_relative_paths
because it would break existing projects that use_relative_paths but
hardcoded the prefix for hooks. Instead we add a second boolean,
use_relative_hooks that triggers the behavior.
Adds tests for the new behavior and a test for existing interactio
between hooks and recursedeps.
BUG=chromium:875245
Change-Id: Ie4c526baa425ff887b3be54e0feca7c597ededec
Reviewed-on: https://chromium-review.googlesource.com/1213327
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
gclient setdep used to assume that the version of a CIPD package was always
of the form 'version:<something>'.
However, the following are all valid:
* SHA1 hashes that specify a single instance
* tags (tagname:tagvalue)
* References (e.g. 'latest') that are dynamically resolved by the CIPD server.
This change adds support for all of them:
* GetCIPD simply returns whatever the 'version' entry is, without assuming
it begins with 'version:'.
* SetCIPD sets 'version' ta whatever it is passed, without prepending 'version:'
at the beginning.
Bug: 858978
Change-Id: I53669c5df7fb51cde42e0af271139b5719a47622
Reviewed-on: https://chromium-review.googlesource.com/1120943
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This is a reland of ff62224443
should_process was set to None in the previous attempt, so CIPD dependencies were not processed.
This CL fixed that.
Original change's description:
> Reland "Expand variables in gclient flattened output."
>
> This is a reland of a32f98e652
>
> Original change's description:
> > Expand variables in gclient flattened output.
> >
> > Bug: 848990
> > Change-Id: I0ad7e4f965973edbc5a335bd30f9cbd7b04abff2
> > Reviewed-on: https://chromium-review.googlesource.com/1085996
> > Reviewed-by: Michael Moss <mmoss@chromium.org>
> > Reviewed-by: Aaron Gable <agable@chromium.org>
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
>
> Tbr: agable@chromium.org
> Bug: 848990
> Change-Id: I7843544b79b2ab7e2046c187d13ea3eb65fc1b7d
> Reviewed-on: https://chromium-review.googlesource.com/1085975
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Bug: 848990
Change-Id: Ic804be1b84bf8402a741a4189b60372075dfb6f3
Reviewed-on: https://chromium-review.googlesource.com/1087368
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This is done in gclient_eval, so we can remove all code
that deals with deps_os and hooks_os from gclient.
Bug: 839925
Change-Id: I491819207a712d62008ff010e313add87d22c937
Reviewed-on: https://chromium-review.googlesource.com/1058375
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Previously, 'gclient sync' would process "gn_args" settings in any
recursed DEPS files, potentially producing multiple output files or
conflicting output files, but 'gclient flatten' would only ever include
one set of "gn_args" settings in the flattened output, and only if they
occurred in the top-level DEPS file.
This makes 'gclient sync' and 'gclient flatten' more consistent by
restricting them both to a single instance of those setting, and
requiring those setting to be defined either in the top-level DEPS file,
or in a recursedeps file specificed by the new 'gclient_gn_args_from'
setting.
R=dpranke@google.com, ehmaldonado@google.com
Bug: 825063
Change-Id: If90d952e47367c50b36daade16a26b29aec0c9db
Reviewed-on: https://chromium-review.googlesource.com/1039870
Reviewed-by: Michael Moss <mmoss@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Michael Moss <mmoss@chromium.org>
Introduce a getdep command to get revision/version/variable value information
as a counterpart for gclient setdep.
It will be useful for autorollers.
Bug: 760633
Change-Id: Iabeae0e78c6fbdcb1a3a79cfb380ac2d83f256d5
Reviewed-on: https://chromium-review.googlesource.com/999171
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
If a dependency was unpinned, support adding a revision to it.
R=agable@chromium.org
Bug: 760633
Change-Id: Id2c9fe5174458acaea334726176b88558425ef5a
Reviewed-on: https://chromium-review.googlesource.com/998735
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Now we respect comments before the first variable.
Bug: 760633
Change-Id: Ibe60d719429c033415bfb1c99942c9d04601d967
Reviewed-on: https://chromium-review.googlesource.com/998683
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
If the variable doesn't already exist, gclient setdep will attempt to
create a new variable with the given name and value as the first variable
in the vars dict.
R=agable@chromium.org, mmoss@chromium.org
Bug: 760633
Change-Id: I2462e70c3695a730f87c58e56c639104efbfa54a
Reviewed-on: https://chromium-review.googlesource.com/989282
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
This is useful for processing purely "local" DEPS files, with a command-line like:
gclient revinfo --output-json /tmp/out \
--spec 'solutions=[{"name": ".", "deps_file": "myDEPS", "url": None}]'
This is specifically to fix the "revinfo --output-json" case, but it
also cleans up some errors in the 'flattening' test cases. Note that
None values already work when using 'revinfo' with stdout output, it's
just the json output that complains.
BUG=825063
Change-Id: If5f69ba51cb9d0aa0f2b48a2f9b4ed8706b4c738
Reviewed-on: https://chromium-review.googlesource.com/989022
Commit-Queue: Michael Moss <mmoss@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Add support for more ways to express revisions in DEPS files, notably
"<origin-url>@<revision>" which is needed for src-internal.
Bug: 760633
Change-Id: I86724b9c077c6101a0c424d80460123d15533da8
Reviewed-on: https://chromium-review.googlesource.com/988493
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
This is a reland of 88f9c40e0c
We no longer need the vars dict to be declared before vars can be used.
This was causing problems because gclient flatten printed vars after deps,
breaking the above assumption.
Original change's description:
> gclient eval: Expand vars while parsing DEPS files
>
> Introduce a Parse function that takes care of expanding vars while parsing
> the DEPS file.
>
> It wraps Exec and exec calls, and supports deferring the expansion until
> later, so gclient flatten gets access to the unexpanded version.
>
> Bug: 821199
> Change-Id: I943b021cc4474c9cda67b3816b841dd8ada3f5b2
> Reviewed-on: https://chromium-review.googlesource.com/973749
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Bug: 821199
Change-Id: I583df23558f91871e1a2aa2574c20d35e54635f6
Reviewed-on: https://chromium-review.googlesource.com/981086
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
This is a reland of 88f9c40e0c
Original change's description:
> gclient eval: Expand vars while parsing DEPS files
>
> Introduce a Parse function that takes care of expanding vars while parsing
> the DEPS file.
>
> It wraps Exec and exec calls, and supports deferring the expansion until
> later, so gclient flatten gets access to the unexpanded version.
>
> Bug: 821199
> Change-Id: I943b021cc4474c9cda67b3816b841dd8ada3f5b2
> Reviewed-on: https://chromium-review.googlesource.com/973749
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Bug: 821199
Change-Id: Ic04af0cae59abc01a0382e2de3497a91cf7e62fd
Reviewed-on: https://chromium-review.googlesource.com/978561
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This reverts commit 88f9c40e0c.
Reason for revert: suspected to be causing update scripts failures and prevent gce bots from connecting
https://build.chromium.org/deprecated/tryserver.chromium.win/builders/win7_chromium_rel_ng/builds/128479/steps/update_scripts/logs/stdio
Original change's description:
> gclient eval: Expand vars while parsing DEPS files
>
> Introduce a Parse function that takes care of expanding vars while parsing
> the DEPS file.
>
> It wraps Exec and exec calls, and supports deferring the expansion until
> later, so gclient flatten gets access to the unexpanded version.
>
> Bug: 821199
> Change-Id: I943b021cc4474c9cda67b3816b841dd8ada3f5b2
> Reviewed-on: https://chromium-review.googlesource.com/973749
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>
TBR=agable@chromium.org,dpranke@chromium.org,ehmaldonado@chromium.org
Change-Id: Ib9b84c13ef9b56735fd8f714b74a6601c61715e5
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 821199
Reviewed-on: https://chromium-review.googlesource.com/976721
Reviewed-by: Benjamin Pastene <bpastene@chromium.org>
Commit-Queue: Benjamin Pastene <bpastene@chromium.org>
Introduce a Parse function that takes care of expanding vars while parsing
the DEPS file.
It wraps Exec and exec calls, and supports deferring the expansion until
later, so gclient flatten gets access to the unexpanded version.
Bug: 821199
Change-Id: I943b021cc4474c9cda67b3816b841dd8ada3f5b2
Reviewed-on: https://chromium-review.googlesource.com/973749
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Since we only support Var, and Var only does one thing, we hard code that
into the parser. Then, we don't need global_scope.
local_scope hasn't been needed for a while.
Bug: 760633
Change-Id: I21b171a8b71e7dcaf8e29c780ca88b9f46f368e8
Reviewed-on: https://chromium-review.googlesource.com/972611
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This is a reland of 7f4c905fc5
Original change's description:
> gclient: Add commands to edit dependencies and variables in DEPS
>
> Adds 'gclient setvar' and 'gclient setdep' commands to edit variables
> and dependencies in a DEPS file.
>
> Bug: 760633
> Change-Id: I6c0712cc079dbbbaee6541b7eda71f4b4813b77b
> Reviewed-on: https://chromium-review.googlesource.com/950405
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>
Bug: 760633
Change-Id: Ia46c74d02e5cc3b67517dfa248f597cb3d98ef3d
Reviewed-on: https://chromium-review.googlesource.com/969457
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
This reverts commit 7f4c905fc5.
Reason for revert:
When running "gclient sync" on a V8 checkout, it says:
File "/work/chrome/depot_tools/gclient.py", line 995, in run
self.ParseDepsFile()
File "/work/chrome/depot_tools/gclient.py", line 874, in ParseDepsFile
self._postprocess_deps(deps, rel_prefix), use_relative_paths)
File "/work/chrome/depot_tools/gclient.py", line 660, in _postprocess_deps
dval['condition'], self.condition)
TypeError: '_NodeDict' object does not support item assignment
on the recursive descent into third_party/android_tools/DEPS. Any chance that's related to https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/950405 ?
Original change's description:
> gclient: Add commands to edit dependencies and variables in DEPS
>
> Adds 'gclient setvar' and 'gclient setdep' commands to edit variables
> and dependencies in a DEPS file.
>
> Bug: 760633
> Change-Id: I6c0712cc079dbbbaee6541b7eda71f4b4813b77b
> Reviewed-on: https://chromium-review.googlesource.com/950405
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>
TBR=agable@chromium.org,ehmaldonado@chromium.org
Change-Id: If58f6b15d31b19fc53294f1e41d26b4e684a2cf9
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 760633
Reviewed-on: https://chromium-review.googlesource.com/969165
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Adds 'gclient setvar' and 'gclient setdep' commands to edit variables
and dependencies in a DEPS file.
Bug: 760633
Change-Id: I6c0712cc079dbbbaee6541b7eda71f4b4813b77b
Reviewed-on: https://chromium-review.googlesource.com/950405
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Bug: 570091
Change-Id: Iae9dad68a75d751ceac6379baac588f32c59aa06
Reviewed-on: https://chromium-review.googlesource.com/548935
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
This makes it possible to run hooks properly
in flattened DEPS.
Bug: 570091
Change-Id: If8175a57ebe8f607bd4ac83d4a26dcc4cc18165c
Reviewed-on: https://chromium-review.googlesource.com/535476
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
This way we can get e.g. ordered dict as needed for conditions.
Only the new logic does it, not the regular python exec logic.
Bug: 570091
Change-Id: Ia5554e5b018085b3b9bd876b7f28a9f8e54a7984
Reviewed-on: https://chromium-review.googlesource.com/522564
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
It's still used in some places like CDM.
This fixes the bug below.
Bug: 728343
Change-Id: I0f9e2bbd0339838ee6d1d240c584858947cf36d2
Reviewed-on: https://chromium-review.googlesource.com/522682
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Reviewed-by: Emma Soederberg <emso@google.com>
This is a step towards implementing conditions.
Bug: 570091
Change-Id: I99467033082d96021854c23dcff3fc2b56f995b4
Reviewed-on: https://chromium-review.googlesource.com/517107
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
This should make error messages more friendly and easier to address.
Bug: 570091
Change-Id: Id43d6ecdc5521b07fbe684b6fdbcc09f6fbc6448
Reviewed-on: https://chromium-review.googlesource.com/513907
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Edward Lemur
James Darpinian
Ben Pastene
Aaron Gable
Raul Tambre
Corentin Wallez
John Budorick
Michael Moss
Egor Pasko
Dirk Pranke
Paweł Hajdan, Jr