Commit Graph

703 Commits (b8164180d2c339940c99db74344c865e67ca064d)

Author SHA1 Message Date
Aravind Vasudevan b8164180d2 Warn when fsmonitor is enabled and git submodules are used
Change-Id: I181bf180f762282d5b4bc614d12a91125f56e063
Bug: 1475405
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4814429
Auto-Submit: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Aravind Vasudevan ed935cf29b Remove deadcode from depot_tools scripts
Removing some deadcode from depot_tools scripts. More follow-up CLs will
be made after this to remove more dead code.

R=sokcevic,gavinmak

Change-Id: Iad20e9e1737e49c1370248d32c6317cf2be63de4
Bug: 1475776
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4811216
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2 years ago
Gavin Mak 65c49b18b9 Drop py2 support in gclient and related files
python3 is the only supported version of python in depot_tools.

Bug: 1475402
Change-Id: I17174d7252b5cbf698700333d3cd561c6591d0a1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4809190
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Josip Sokcevic 293aa65afa [gclient] Fix gitmodule for Windows machine
Use universal newlines when writing to gitmodules. Also, split
update-index command so we don't hit Windows limit for process lenght.

R=jojwang

Bug: 1475328
Change-Id: Ia9bba4620a276c31081a9773bac8f6bd309bb86e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4808507
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
2 years ago
Josip Sokcevic c9dae6458f [gclient] Optimize gitmodules script
This reduces calls to git-update-index from N (number of deps) to 1.

R=aravindvasudev@google.com, jojwang@google.com

Change-Id: I46d7eec37d8e27cd54039ff0a03782857b440317
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4795126
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Josip Sokcevic 5561f8be12 [gclient] Fix setdep if relative path is not used
R=aravindvasudev@google.com, jojwang@google.com

Change-Id: Ic4f7046ac4e1a22a82db97466c43afee99c393de
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4795232
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Joanna Wang 978f43dd52 Use git ls-files -s to include gitlinks from staged patches.
Bug: 1471685
Change-Id: I3fdf9630ed0e55dc10a1c578c8e052f76461e800
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4776220
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Aravind Vasudevan f44f219157 Change cwd when --deps-file is provided
When `gclient setdep` has `--deps-file` arg provided, use its basedir to
run submodule commands. And fail gracefully when thsoe commands throw
exceptions.

Change-Id: I94192b20fb5c34729c75404a34762546f6117689
Bug: 1472449
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4775173
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Joanna Wang e460dac109 Set default value newly gitlinks from applied patches.
Bug: 1471685
Change-Id: I850778f030f77bda746227c3e2fde4bbaee55a3c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4775447
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Auto-Submit: Joanna Wang <jojwang@chromium.org>
2 years ago
Aravind Vasudevan c91e2aaa9c Fix filepath issue within ParseGitSubmodules()
The filepath already has cwd appended so the command doesn't need cwd.

Change-Id: I20c6fea27aee963e90cdb8a15cb087771dafa0fa
Bug: 1472143
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4771942
Commit-Queue: Scott Lee <ddoman@chromium.org>
Auto-Submit: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Scott Lee <ddoman@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
2 years ago
Josip Sokcevic de6bc6620a Improve presubmit check messaging
Also add oneline status on gclient gitmodules.

R=jojwang@google.com

Change-Id: I05c9f856ce6fd1c3ebf1dc7da672d25196a4cb67
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4771975
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Joanna Wang 11b07527e6 Sync gitlinks to DEPS so commits from applied patches are used.
Bug: 1471685
Change-Id: If9ec3178b7e265dd7da8b8bc672131d33ccd36a5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4764134
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
2 years ago
Aravind Vasudevan e29d5e9ad5 Fix ParseGitSubmodules for git v2.34
The `--format` option for `git ls-tree` was introduced in git v2.36 and
hence gclient breaks for lower git versions.

Change-Id: I02750904aa78bb5fd5f3f57dc2ad42e355fa4a42
Bug: 1471241
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4765682
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
2 years ago
Aravind Vasudevan 18c726d5b1 Fix git submodule status pattern
The ParseGitModules() uses `git submodule status` result to get the
commit hashes for the submodules. It assumes the command's result to not
have a `git describe` at the end of each line, and hence only works in
certain scenarios. This change fixes this by updating the parser.

Change-Id: I8f4e3fae6562dafd0e127255a8f26e7336a75683
Bug: 1471241
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4763348
Auto-Submit: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Joanna Wang 3ab2f21289 gclient: add gitmodules command to update/add submodules based on DEPS.
Bug: 1421776
Change-Id: Id1ac48c4b65c17027fa24d0ba350a1a7f2eec64d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4665787
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Joanna Wang 59e1011541 Fix hook base paths when repo is not relative but parent is.
This makes hooks path behavior match deps path behavior.

Bug: 1468658
Change-Id: Ib6685cb52679249b389599ada549e2c4410e812c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4729084
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
2 years ago
Josip Sokcevic e7a7a90a24 Fix gclient read submodule path
We expect gclient to place git submodules in the same directory where
.gitmodules is, and without any modification.

Prior to this patch, gclient reads paths and use them as-is. So, if a
project ./foo/ has submodule 'bar', the git repository will be placed in
./bar instead of ./foo/bar iff use_relative_paths is set to false.

Bug: 1429149
Change-Id: I4f99bb11a20d9779c1226926187ebfd8f5d829a6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4680282
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Josip Sokcevic 9d18c9eb65 Reland "Drop support for py2 presubmit hook win mapping"
This is a reland of commit e27d7c3816

Original change's description:
> Drop support for py2 presubmit hook win mapping
>
> We no longer support py2 so we shouldn't have a special handling for py2
> hooks.
>
> R=gavinmak@google.com
>
> Change-Id: I038f462a4991d17bedb3e02104374b3ad1ffa5ff
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4541664
> Commit-Queue: Gavin Mak <gavinmak@google.com>
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>

Change-Id: I190fb7c1ee4e76adfe74196c1ed579a424314a02
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4669591
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Aravind Vasudevan 11d847f07b Fix setdep to run git command only for submodules is enabled
Change-Id: If748b0cb63e35fe68b608ccdb147a1728d3ae42c
Bug: 1463007
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4671611
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Aravind Vasudevan af456fdb7f Read git submodules within gclient
Design Doc: go/depot-tools-on-submodules
Change-Id: Ic7025a26ace8cb0b46d915909ee1321c6d044797
Bug: 1429149
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4599672
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
2 years ago
Aravind Vasudevan cb8023d987 Write git dependencies to submodules
Design Doc: go/depot-tools-on-submodules
Change-Id: I6cccfbb78b96bfcd064939c422fb9da624aae335
Bug: 1429149
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4599668
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
2 years ago
Aravind Vasudevan b6eaed26fc Add git_dependencies flag to DEPS
This change adds `git_dependencies` flag to the DEPS. This will be used to track the submodule migration state.

Design Doc: go/depot-tools-on-submodules
Change-Id: I547ad8ebbce2535960d4b6e00c0b14c00118f544
Bug: 1429149
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4591777
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
2 years ago
Robbie Iannucci 3db3276489 Revert "loongarch64: Add loongarch64 architecture detection support."
This reverts commit 0648830632.

Reason for revert: Pending clarity on https://bugs.chromium.org/p/chromium/issues/detail?id=1361860

Original change's description:
> loongarch64: Add loongarch64 architecture detection support.
>
> Bug: 1361860
> Change-Id: I432208fd78fb99140a97b3979a257f4e2e513807
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3886536
> Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
> Reviewed-by: Brian Ryner <bryner@google.com>
> Reviewed-by: Gavin Mak <gavinmak@google.com>

Bug: 1361860
Change-Id: Icf9f88a7173d4d403f4846fe01342794a0f5db9e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4667669
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
2 years ago
Wang Qing 0648830632 loongarch64: Add loongarch64 architecture detection support.
Bug: 1361860
Change-Id: I432208fd78fb99140a97b3979a257f4e2e513807
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3886536
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Brian Ryner <bryner@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2 years ago
Josip Sokcevic d47a9c2c46 Revert "Allow patching multiple directories if shared URL"
This reverts commit 8559499c4c.

Reason for revert: breaks internal presubmits
https://crbug.com/1457050

Original change's description:
> Allow patching multiple directories if shared URL
>
> If there are more than one DEPS entry that can be patched (e.g. two DEPS
> entries with the same URL, but different path, ie. src-internal and
> src/internal), then gclient should patch all of them and not just the
> first one it seees.
>
> R=gavinmak@google.com
>
> Change-Id: I617b1771062a718421d8ed17941bc3eef3288005
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4630529
> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
> Reviewed-by: Gavin Mak <gavinmak@google.com>

Bug: 1457050
Change-Id: I76fd217917a569bc3cc7220c2d2090675996dcdc
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4635526
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
2 years ago
Josip Sokcevic 8559499c4c Allow patching multiple directories if shared URL
If there are more than one DEPS entry that can be patched (e.g. two DEPS
entries with the same URL, but different path, ie. src-internal and
src/internal), then gclient should patch all of them and not just the
first one it seees.

R=gavinmak@google.com

Change-Id: I617b1771062a718421d8ed17941bc3eef3288005
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4630529
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2 years ago
Josip Sokcevic 7b5e3d7868 Set exit code on gclient getdep if rev is invalid
Add exit code if requested DEP is not found. This way, machines
don't need to parse stderr and make decision; instead they can use exit
code.

R=jojwang@google.com

Bug: 1454296
Change-Id: I73edda333342c2ebf78eb0e40b73cb87562989c5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4608990
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
2 years ago
Dan Le Febvre b0e8e7adbc output actual CIPD package URL instead of 'None' when calling revinfo -a
Note: we are only evaluating cipd variables in the URL not the package string on the left hand side, as to not break any existing expected behaviour.

Bug:b/279097790
Change-Id: I0b6008254bd387584b9c0ee083c79bf5dc77fbc8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4536475
Commit-Queue: Dan Le Febvre <dlf@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2 years ago
Michael Spang 3c7d6f19dc Revert "Drop support for py2 presubmit hook win mapping"
This reverts commit e27d7c3816.

Reason for revert: Retroactively broke branch builds for Nest

Original change's description:
> Drop support for py2 presubmit hook win mapping
>
> We no longer support py2 so we shouldn't have a special handling for py2
> hooks.
>
> R=gavinmak@google.com
>
> Change-Id: I038f462a4991d17bedb3e02104374b3ad1ffa5ff
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4541664
> Commit-Queue: Gavin Mak <gavinmak@google.com>
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>

Change-Id: Id4f0f3950d41a5829b279c4844f02c2a6ccf41fe
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4544429
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Michael Spang <spang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Josip Sokcevic e27d7c3816 Drop support for py2 presubmit hook win mapping
We no longer support py2 so we shouldn't have a special handling for py2
hooks.

R=gavinmak@google.com

Change-Id: I038f462a4991d17bedb3e02104374b3ad1ffa5ff
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4541664
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Alexander Thomas 24996afd00 Revert "Reland "resolve CIPD package names in gclient""
This reverts commit d69302761b.

Reason for revert: Broke Dart CI
```
gclient.py revinfo --output-json output.json
   --filter='https://chrome-infra-packages.appspot.com/dart/dart-sdk/${platform}'
```

No longer returns:
```
 {
   "sdk/tools/sdks/dart-sdk:dart/dart-sdk/${platform}": {
     "rev": "git_revision:7a6514d1377175decd3a886fe4190fbbebddac3a", 
     "url": "https://chrome-infra-packages.appspot.com/dart/dart-sdk/${platform}"
   }
 }
```

Bad build:
https://ci.chromium.org/ui/p/dart/builders/ci.sandbox/vm-aot-msan-linux-release-x64/90/overview
Good build:
https://ci.chromium.org/ui/p/dart/builders/ci.sandbox/vm-aot-msan-linux-release-x64/89/overview

Original change's description:
> Reland "resolve CIPD package names in gclient"
>
> This CL relands the below CL with the addition of handling situations where the CIPD package name has a variable=value and does not equate to true. In this case just return the original value in DEPS.
>
> This is a reland of commit 8faa5514ec
>
> Original change's description:
> > resolve CIPD package names in gclient
> >
> > Currently the package names are not resolved in gclient output, see example below.
> >
> > This is part of broader work to improve CIPD output, the next CL will replace 'None'.
> >
> > ```
> > $~ gclient revinfo -a | grep '${platform}\|${arch}'
> >
> > src/buildtools/linux64:gn/gn/linux-${arch}: None
> > src/buildtools/reclient:infra/rbe/client/${platform}: None
> > src/third_party/dawn/third_party/ninja:infra/3pp/tools/ninja/${platform}: None
> > src/third_party/dawn/tools/golang:infra/3pp/tools/go/${platform}: None
> > src/third_party/devtools-frontend-internal/devtools-frontend/third_party/esbuild:infra/3pp/tools/esbuild/${platform}: None
> > src/third_party/devtools-frontend/src/third_party/esbuild:infra/3pp/tools/esbuild/${platform}: None
> > src/third_party/ninja:infra/3pp/tools/ninja/${platform}: None
> > src/tools/luci-go:infra/tools/luci/isolate/${platform}: None
> > src/tools/luci-go:infra/tools/luci/swarming/${platform}: None
> > src/tools/resultdb:infra/tools/result_adapter/${platform}: None
> > ```
> >
> > Bug:b/279097790
> > Change-Id: I21abf2579910e9d79d9ee0dcd018ee761e3d3c1c
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4463983
> > Reviewed-by: Rachael Newitt <renewitt@google.com>
> > Reviewed-by: Gavin Mak <gavinmak@google.com>
> > Commit-Queue: Dan Le Febvre <dlf@google.com>
>
> Bug: b/279097790
> Change-Id: Ib08dac506ce221243c595dde5cb00e8e0d7dc7ed
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4501582
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Dan Le Febvre <dlf@google.com>

Bug: b/279097790
Change-Id: Ibf82354ac5005d9d6279d88aa99c8fb344aa6833
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4518113
Owners-Override: Daniel Cheng <dcheng@chromium.org>
Commit-Queue: Daniel Cheng <dcheng@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Dan Le Febvre <dlf@google.com>
2 years ago
Dan Le Febvre d69302761b Reland "resolve CIPD package names in gclient"
This CL relands the below CL with the addition of handling situations where the CIPD package name has a variable=value and does not equate to true. In this case just return the original value in DEPS.

This is a reland of commit 8faa5514ec

Original change's description:
> resolve CIPD package names in gclient
>
> Currently the package names are not resolved in gclient output, see example below.
>
> This is part of broader work to improve CIPD output, the next CL will replace 'None'.
>
> ```
> $~ gclient revinfo -a | grep '${platform}\|${arch}'
>
> src/buildtools/linux64:gn/gn/linux-${arch}: None
> src/buildtools/reclient:infra/rbe/client/${platform}: None
> src/third_party/dawn/third_party/ninja:infra/3pp/tools/ninja/${platform}: None
> src/third_party/dawn/tools/golang:infra/3pp/tools/go/${platform}: None
> src/third_party/devtools-frontend-internal/devtools-frontend/third_party/esbuild:infra/3pp/tools/esbuild/${platform}: None
> src/third_party/devtools-frontend/src/third_party/esbuild:infra/3pp/tools/esbuild/${platform}: None
> src/third_party/ninja:infra/3pp/tools/ninja/${platform}: None
> src/tools/luci-go:infra/tools/luci/isolate/${platform}: None
> src/tools/luci-go:infra/tools/luci/swarming/${platform}: None
> src/tools/resultdb:infra/tools/result_adapter/${platform}: None
> ```
>
> Bug:b/279097790
> Change-Id: I21abf2579910e9d79d9ee0dcd018ee761e3d3c1c
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4463983
> Reviewed-by: Rachael Newitt <renewitt@google.com>
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Dan Le Febvre <dlf@google.com>

Bug: b/279097790
Change-Id: Ib08dac506ce221243c595dde5cb00e8e0d7dc7ed
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4501582
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Dan Le Febvre <dlf@google.com>
2 years ago
Dan Le Febvre f2da906faf Revert "resolve CIPD package names in gclient"
This reverts commit 8faa5514ec.

Reason for revert: cipd expand-package-name errors on wrong platform

Original change's description:
> resolve CIPD package names in gclient
>
> Currently the package names are not resolved in gclient output, see example below.
>
> This is part of broader work to improve CIPD output, the next CL will replace 'None'.
>
> ```
> $~ gclient revinfo -a | grep '${platform}\|${arch}'
>
> src/buildtools/linux64:gn/gn/linux-${arch}: None
> src/buildtools/reclient:infra/rbe/client/${platform}: None
> src/third_party/dawn/third_party/ninja:infra/3pp/tools/ninja/${platform}: None
> src/third_party/dawn/tools/golang:infra/3pp/tools/go/${platform}: None
> src/third_party/devtools-frontend-internal/devtools-frontend/third_party/esbuild:infra/3pp/tools/esbuild/${platform}: None
> src/third_party/devtools-frontend/src/third_party/esbuild:infra/3pp/tools/esbuild/${platform}: None
> src/third_party/ninja:infra/3pp/tools/ninja/${platform}: None
> src/tools/luci-go:infra/tools/luci/isolate/${platform}: None
> src/tools/luci-go:infra/tools/luci/swarming/${platform}: None
> src/tools/resultdb:infra/tools/result_adapter/${platform}: None
> ```
>
> Bug:b/279097790
> Change-Id: I21abf2579910e9d79d9ee0dcd018ee761e3d3c1c
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4463983
> Reviewed-by: Rachael Newitt <renewitt@google.com>
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Dan Le Febvre <dlf@google.com>

Bug: b/279097790
Change-Id: I0beba8606d8cceeb815629d6eae6d8021c7731a8
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4501579
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Auto-Submit: Dan Le Febvre <dlf@google.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Owners-Override: Ben Pastene <bpastene@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
2 years ago
Dan Le Febvre 8faa5514ec resolve CIPD package names in gclient
Currently the package names are not resolved in gclient output, see example below.

This is part of broader work to improve CIPD output, the next CL will replace 'None'.

```
$~ gclient revinfo -a | grep '${platform}\|${arch}'

src/buildtools/linux64:gn/gn/linux-${arch}: None
src/buildtools/reclient:infra/rbe/client/${platform}: None
src/third_party/dawn/third_party/ninja:infra/3pp/tools/ninja/${platform}: None
src/third_party/dawn/tools/golang:infra/3pp/tools/go/${platform}: None
src/third_party/devtools-frontend-internal/devtools-frontend/third_party/esbuild:infra/3pp/tools/esbuild/${platform}: None
src/third_party/devtools-frontend/src/third_party/esbuild:infra/3pp/tools/esbuild/${platform}: None
src/third_party/ninja:infra/3pp/tools/ninja/${platform}: None
src/tools/luci-go:infra/tools/luci/isolate/${platform}: None
src/tools/luci-go:infra/tools/luci/swarming/${platform}: None
src/tools/resultdb:infra/tools/result_adapter/${platform}: None
```

Bug:b/279097790
Change-Id: I21abf2579910e9d79d9ee0dcd018ee761e3d3c1c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4463983
Reviewed-by: Rachael Newitt <renewitt@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Dan Le Febvre <dlf@google.com>
2 years ago
Aravind Vasudevan 3d760ccab5 Revert "Use default number of cores for gclient jobs"
This reverts commit fe5a0b1289.

Reason for revert: gclient sync hangs for several MacOS users.
Original change's description:
> Use default number of cores for gclient jobs
>
> Instead of always setting jobs to 1 for ARM (which includes Mac), use
> number of cores. Some old boards have not worked well with parallel
> sync, but I don't have historical context was going on. My assumption is
> that those boards were single-core.
>
> See: https://codereview.chromium.org/10116033
>
> R=​aravindvasudev@google.com
>
> Bug: 1410757
> Change-Id: I0b7e3363375ba719c73d4d7141e6cab34bf7d32f
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4198732
> Reviewed-by: Dirk Pranke <dpranke@google.com>
> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>

Bug: 1410757, 1427050
Change-Id: Ifde8393de047f63a448dbc1ef49a2f8049f49331
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4383325
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Auto-Submit: Aravind Vasudevan <aravindvasudev@google.com>
2 years ago
sokcevic 71b606ecff Fix use_relative_path on Windows
DEPS entries all have slashes as their path separators.

If use_relative_path is set to true, .gclient_entries will actually
contain backslashes instead slashes. This is problematic when DEPS
file transitions from using use_relative_path = False to True.

In such case, gclient thinks there are two distinct repositories
and it will delete one with slashes. As this operation is the last
step of gclient sync, it results in affected repositories being
wrongfully removed.

This change forces all deps entries to have slashes regardless of
operating system.

R=gavinmak@google.com

Bug: 1422665
Change-Id: Idb73510d911a260bccf63d24a8d3452998b147f7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4348792
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Joanna Wang 9144b67c7f gclient: Create graphviz file in flatten command.
Change-Id: I57808ecb43df51a007945874339400fee2342807
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4229571
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
2 years ago
Josip Sokcevic fe5a0b1289 Use default number of cores for gclient jobs
Instead of always setting jobs to 1 for ARM (which includes Mac), use
number of cores. Some old boards have not worked well with parallel
sync, but I don't have historical context was going on. My assumption is
that those boards were single-core.

See: https://codereview.chromium.org/10116033

R=aravindvasudev@google.com

Bug: 1410757
Change-Id: I0b7e3363375ba719c73d4d7141e6cab34bf7d32f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4198732
Reviewed-by: Dirk Pranke <dpranke@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Jonas Termansen bf7eb5292c [portability] Support unknown operating systems in gclient.
Detect the name of unknown operating systems using uname, if available,
since it doesn't append the operating system version.

Change-Id: Idab7bd0db65a8d424ec2fd48f06247405b6649e4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4169240
Auto-Submit: Jonas Termansen <sortie@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Josip Sokcevic 1b8211ff13 Restore git files that may be deleted by CIPD
When CIPD removes a package, it removes the files. However, it's
possible that those files were checked in beforehand (for example,
https://crrev.com/c/3858186).
bot_update/gclient sync will first run git operations followed by cipd
ensure.

This patch restores any files that may have been deleted after cipd
ensure.

Bug: 1369452
Change-Id: I43ff9553118a20ed250bddc15153f02828847ed3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3928237
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
3 years ago
Joanna Wang 0187079c09 [no-sync] Save previous values to files.
Tested locally with chromium/src.
going from 17s -> <1s

Bug: 1339472
Change-Id: I71b90dcd6a7934ea7c9722dd040d63645792078e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3791746
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
3 years ago
Arthur Milchior 08cd5fec1f Clarify documentation from `gclient recurse`
Applying a command on a value usually mean to call `cmd value`. Hence
it seemed important to me to clarify that the current working
directory is changed.

Tested:
* Entered `gclient recurse --help` and checked the new message appear.
* Entered both examples in the terminal and checked the output.

Thanks to sdefresne for his help understanding recurse.

Change-Id: I729efc2014ed3cf90112cc89875a283d58ce8af0
Fixed: 1345272
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3769966
Reviewed-by: Arthur Milchior <arthurmilchior@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Arthur Milchior <arthurmilchior@google.com>
3 years ago
Joanna Wang a84a16b863 [no-sync] Set _should_sync and add flag to control if the experiment should be enabled.
Bug:1339472
Change-Id: I19abca1f4319a89cc461935a83d136c8870944dc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3721601
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
3 years ago
Joanna Wang f3edc50445 [no-sync] Store previous sync commit in memory.
- Instead of expecting the caller to run `rev-parse HEAD` before
  calling gclient, we can just store the latest commit on disk
  - this gets around the problem where gclient might git reset to the
     base checkout after applying patch-refs, so calling rev-parse
     before calling gclient sync might not actually return the latest
     commit of the last sync.
- also moving os.environ[PREVIOUS_CUSTOM_VARS] setting to earlier when
  we only have the solutions in self.dependencies because all other
  dependencies inherit from the solutions so there's no point storing
  custom_vars for the rest.

Bug: 1339472
Change-Id: I6a3570f09153bd8087bbe6bdab7ece4949856aae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3750491
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
3 years ago
Joanna Wang 18af7efdd8 [no-sync] Update ParseDepsFile to process deps and hooks appropriately with should_sync.
Bug: 1339472
Change-Id: Iea077cdb655105d27431ff4f677d93eec121bc31
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3738361
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
3 years ago
Joanna Wang 6628661eba [no-sync] Add a skip_sync_revisions and process it before running deps.
Bug: 1339472
Change-Id: I429489f7deea035c47e27e32ada858fb8a827643
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3735487
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
3 years ago
Joanna Wang 4e6264cc28 [gclient] Specify which DEPS is still using `use_relative_hooks`.
Bug:1107325, 1340011
Change-Id: I90bce90cba2ccf2a37b703b9bad49b5f79cfde33
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3738359
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
3 years ago
Joanna Wang 0a2356cd39 Remove redundant should_process check.
AFAICT
- the code never reaches this point because we already check should process before adding it to the queue. http://shortn/_PWFae17qw4

- this might be necessary if the value of should_process changes between adding it to the queue and running it. But I don't see that happening anywhere. should_process is only ever set and modified here: http://shortn/_eirozAoIm4 which happens during in the parent's run() before the deps get added to the queue.

The should_process check that happens at the beginning of run() was added here: https://codereview.chromium.org/3124017/diff/2001/gclient.py

The should_process check before a deps gets enqueued was added later here: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/559150/

where several should_process checks were added and I think the author just didn't realize that, while the other methods might need them, run() already had a should_process check.

Alternatively, we could remove the should_process check before enqueue-ing. But that would result in enqueuing things that we currently don't enqueue, even if we ultimately don't do anything with it.

Change-Id: Ib3be61310c09a681988baa75fd562c7596e9e044
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3721600
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
3 years ago
Ergün Erdoğmuş 28190a2f35 Fix not creating GNArgs file when there are multiple dependencies
Bug: 1336227
Change-Id: I53a13b452829f280816cc07a0907958348c23d56
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3702359
Commit-Queue: Ergün Erdoğmuş <ergunsh@chromium.org>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
3 years ago
Aravind Vasudevan 810598d1f3 Add protocol_override entry to .gclient
The meta quest browser has an issue with the initial design where the all the dependencies within the same tree are overridden with the solution URL's scheme. This fix updates fetch to add a .gclient entry instead of implicitly overriding the URL's scheme.

Bug: 1336027
Change-Id: Iabe5b9017f6eb9e682a0c721f43dd5f347ffcbfb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3703942
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
3 years ago